python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3153736 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['2571901'] with mtr_portfolio_ids : ['20410114'] and first list_photo_ids : [] new path : /proc/3153736/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 8 ; length of list_pids : 8 ; length of list_args : 8 time to download the photos : 1.466747522354126 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Mon Feb 10 16:40:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10553 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-10 16:40:33.453605: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-10 16:40:33.462059: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-10 16:40:33.463630: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f7774000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-10 16:40:33.463694: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-10 16:40:33.466614: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-10 16:40:33.592097: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x21219110 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-10 16:40:33.592161: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-10 16:40:33.593717: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-10 16:40:33.594186: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-10 16:40:33.597017: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-10 16:40:33.599618: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-10 16:40:33.600088: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-10 16:40:33.602800: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-10 16:40:33.604143: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-10 16:40:33.608817: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-10 16:40:33.610412: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-10 16:40:33.610556: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-10 16:40:33.611387: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-10 16:40:33.611406: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-10 16:40:33.611416: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-10 16:40:33.613054: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9615 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-10 16:40:33.908017: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-10 16:40:33.908201: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-10 16:40:33.908242: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-10 16:40:33.908274: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-10 16:40:33.908307: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-10 16:40:33.908338: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-10 16:40:33.908365: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-10 16:40:33.908392: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-10 16:40:33.910063: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-10 16:40:33.912051: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-10 16:40:33.912146: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-10 16:40:33.912175: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-10 16:40:33.912201: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-10 16:40:33.912227: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-10 16:40:33.912252: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-10 16:40:33.912276: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-10 16:40:33.912302: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-10 16:40:33.913998: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-10 16:40:33.914057: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-10 16:40:33.914074: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-10 16:40:33.914086: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-10 16:40:33.916204: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9615 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-10 16:40:48.682438: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-10 16:40:48.909589: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-10 16:40:50.457475: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-10 16:40:50.458536: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.60G (3865470464 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-10 16:40:50.459562: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.24G (3478923264 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 8 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 2.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 143.60000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 34 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 144.85000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 33 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 2.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 144.35000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 38 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 143.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 45 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 149.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 47 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 142.60000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 39 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 142.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 42 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 142.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 32 Detection mask done ! Trying to reset tf kernel 3154179 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 2059 tf kernel not reseted sub process len(results) : 8 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 8 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10332 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0011172294616699219 nb_pixel_total : 31269 time to create 1 rle with old method : 0.03337883949279785 length of segment : 211 time for calcul the mask position with numpy : 0.004840373992919922 nb_pixel_total : 101689 time to create 1 rle with old method : 0.1095883846282959 length of segment : 485 time for calcul the mask position with numpy : 0.0025110244750976562 nb_pixel_total : 65301 time to create 1 rle with old method : 0.06988835334777832 length of segment : 284 time for calcul the mask position with numpy : 0.0010302066802978516 nb_pixel_total : 33264 time to create 1 rle with old method : 0.03719639778137207 length of segment : 209 time for calcul the mask position with numpy : 0.0013422966003417969 nb_pixel_total : 36362 time to create 1 rle with old method : 0.039476871490478516 length of segment : 266 time for calcul the mask position with numpy : 0.0028824806213378906 nb_pixel_total : 79503 time to create 1 rle with old method : 0.08569455146789551 length of segment : 349 time for calcul the mask position with numpy : 0.0016264915466308594 nb_pixel_total : 37485 time to create 1 rle with old method : 0.04089689254760742 length of segment : 275 time for calcul the mask position with numpy : 0.00047659873962402344 nb_pixel_total : 9603 time to create 1 rle with old method : 0.01027369499206543 length of segment : 127 time for calcul the mask position with numpy : 0.0006833076477050781 nb_pixel_total : 35304 time to create 1 rle with old method : 0.03793978691101074 length of segment : 175 time for calcul the mask position with numpy : 0.0028562545776367188 nb_pixel_total : 75136 time to create 1 rle with old method : 0.08109092712402344 length of segment : 547 time for calcul the mask position with numpy : 0.0024645328521728516 nb_pixel_total : 67689 time to create 1 rle with old method : 0.07290959358215332 length of segment : 479 time for calcul the mask position with numpy : 0.0021462440490722656 nb_pixel_total : 52839 time to create 1 rle with old method : 0.06552529335021973 length of segment : 213 time for calcul the mask position with numpy : 0.00235748291015625 nb_pixel_total : 54086 time to create 1 rle with old method : 0.0625455379486084 length of segment : 515 time for calcul the mask position with numpy : 0.00103759765625 nb_pixel_total : 22143 time to create 1 rle with old method : 0.03096485137939453 length of segment : 213 time for calcul the mask position with numpy : 0.0005128383636474609 nb_pixel_total : 10224 time to create 1 rle with old method : 0.012476444244384766 length of segment : 72 time for calcul the mask position with numpy : 0.0014259815216064453 nb_pixel_total : 30202 time to create 1 rle with old method : 0.03376936912536621 length of segment : 247 time for calcul the mask position with numpy : 0.0015110969543457031 nb_pixel_total : 42782 time to create 1 rle with old method : 0.04709315299987793 length of segment : 240 time for calcul the mask position with numpy : 0.0014426708221435547 nb_pixel_total : 38335 time to create 1 rle with old method : 0.049093008041381836 length of segment : 330 time for calcul the mask position with numpy : 0.0006716251373291016 nb_pixel_total : 20395 time to create 1 rle with old method : 0.023319244384765625 length of segment : 206 time for calcul the mask position with numpy : 0.0006616115570068359 nb_pixel_total : 9737 time to create 1 rle with old method : 0.011429071426391602 length of segment : 128 time for calcul the mask position with numpy : 0.0009095668792724609 nb_pixel_total : 22513 time to create 1 rle with old method : 0.025720834732055664 length of segment : 174 time for calcul the mask position with numpy : 0.0036232471466064453 nb_pixel_total : 80863 time to create 1 rle with old method : 0.092926025390625 length of segment : 473 time for calcul the mask position with numpy : 0.003224611282348633 nb_pixel_total : 76301 time to create 1 rle with old method : 0.09809112548828125 length of segment : 444 time for calcul the mask position with numpy : 0.0013039112091064453 nb_pixel_total : 31221 time to create 1 rle with old method : 0.03464341163635254 length of segment : 277 time for calcul the mask position with numpy : 0.0010671615600585938 nb_pixel_total : 26128 time to create 1 rle with old method : 0.028667449951171875 length of segment : 203 time for calcul the mask position with numpy : 0.0011897087097167969 nb_pixel_total : 27807 time to create 1 rle with old method : 0.029443025588989258 length of segment : 282 time for calcul the mask position with numpy : 0.0019168853759765625 nb_pixel_total : 36307 time to create 1 rle with old method : 0.04024171829223633 length of segment : 300 time for calcul the mask position with numpy : 0.0016617774963378906 nb_pixel_total : 31285 time to create 1 rle with old method : 0.038156986236572266 length of segment : 348 time for calcul the mask position with numpy : 0.0010540485382080078 nb_pixel_total : 16874 time to create 1 rle with old method : 0.01871180534362793 length of segment : 256 time for calcul the mask position with numpy : 0.0008559226989746094 nb_pixel_total : 15694 time to create 1 rle with old method : 0.01754021644592285 length of segment : 263 time for calcul the mask position with numpy : 0.0007913112640380859 nb_pixel_total : 25282 time to create 1 rle with old method : 0.02853846549987793 length of segment : 194 time for calcul the mask position with numpy : 0.002349853515625 nb_pixel_total : 63495 time to create 1 rle with old method : 0.08169722557067871 length of segment : 323 time for calcul the mask position with numpy : 0.003702878952026367 nb_pixel_total : 122418 time to create 1 rle with old method : 0.13587498664855957 length of segment : 631 time for calcul the mask position with numpy : 0.0037713050842285156 nb_pixel_total : 113045 time to create 1 rle with old method : 0.12079119682312012 length of segment : 501 time for calcul the mask position with numpy : 0.0003495216369628906 nb_pixel_total : 7722 time to create 1 rle with old method : 0.008843421936035156 length of segment : 112 time for calcul the mask position with numpy : 0.002147197723388672 nb_pixel_total : 90105 time to create 1 rle with old method : 0.09823131561279297 length of segment : 295 time for calcul the mask position with numpy : 0.001783609390258789 nb_pixel_total : 36755 time to create 1 rle with old method : 0.04138803482055664 length of segment : 223 time for calcul the mask position with numpy : 0.0010037422180175781 nb_pixel_total : 15078 time to create 1 rle with old method : 0.017866849899291992 length of segment : 164 time for calcul the mask position with numpy : 0.0013287067413330078 nb_pixel_total : 38761 time to create 1 rle with old method : 0.04400038719177246 length of segment : 218 time for calcul the mask position with numpy : 0.0013077259063720703 nb_pixel_total : 35849 time to create 1 rle with old method : 0.04010915756225586 length of segment : 277 time for calcul the mask position with numpy : 0.0007216930389404297 nb_pixel_total : 13452 time to create 1 rle with old method : 0.014948606491088867 length of segment : 138 time for calcul the mask position with numpy : 0.0017600059509277344 nb_pixel_total : 42178 time to create 1 rle with old method : 0.04544830322265625 length of segment : 235 time for calcul the mask position with numpy : 0.00229644775390625 nb_pixel_total : 46948 time to create 1 rle with old method : 0.052247047424316406 length of segment : 301 time for calcul the mask position with numpy : 0.002489805221557617 nb_pixel_total : 71593 time to create 1 rle with old method : 0.07755398750305176 length of segment : 313 time for calcul the mask position with numpy : 0.0025708675384521484 nb_pixel_total : 59271 time to create 1 rle with old method : 0.06403470039367676 length of segment : 405 time for calcul the mask position with numpy : 0.0005726814270019531 nb_pixel_total : 9817 time to create 1 rle with old method : 0.01074838638305664 length of segment : 153 time for calcul the mask position with numpy : 0.01633453369140625 nb_pixel_total : 480070 time to create 1 rle with new method : 0.02754807472229004 length of segment : 1096 time for calcul the mask position with numpy : 0.0051555633544921875 nb_pixel_total : 155908 time to create 1 rle with new method : 0.008358001708984375 length of segment : 640 time for calcul the mask position with numpy : 0.0010151863098144531 nb_pixel_total : 29754 time to create 1 rle with old method : 0.03577160835266113 length of segment : 161 time for calcul the mask position with numpy : 0.003648042678833008 nb_pixel_total : 54740 time to create 1 rle with old method : 0.07272171974182129 length of segment : 346 time for calcul the mask position with numpy : 0.0005800724029541016 nb_pixel_total : 13137 time to create 1 rle with old method : 0.013320207595825195 length of segment : 150 time for calcul the mask position with numpy : 0.0012745857238769531 nb_pixel_total : 19357 time to create 1 rle with old method : 0.021149873733520508 length of segment : 347 time for calcul the mask position with numpy : 0.0005269050598144531 nb_pixel_total : 9644 time to create 1 rle with old method : 0.011403560638427734 length of segment : 159 time for calcul the mask position with numpy : 0.0009505748748779297 nb_pixel_total : 17647 time to create 1 rle with old method : 0.020661115646362305 length of segment : 219 time for calcul the mask position with numpy : 0.0015158653259277344 nb_pixel_total : 38468 time to create 1 rle with old method : 0.04237627983093262 length of segment : 236 time for calcul the mask position with numpy : 0.0009982585906982422 nb_pixel_total : 17378 time to create 1 rle with old method : 0.02031874656677246 length of segment : 176 time for calcul the mask position with numpy : 0.0013051033020019531 nb_pixel_total : 32520 time to create 1 rle with old method : 0.03679990768432617 length of segment : 182 time for calcul the mask position with numpy : 0.0009171962738037109 nb_pixel_total : 15669 time to create 1 rle with old method : 0.01734638214111328 length of segment : 136 time for calcul the mask position with numpy : 0.0006852149963378906 nb_pixel_total : 17050 time to create 1 rle with old method : 0.018932580947875977 length of segment : 198 time for calcul the mask position with numpy : 0.0012242794036865234 nb_pixel_total : 25357 time to create 1 rle with old method : 0.026923179626464844 length of segment : 211 time for calcul the mask position with numpy : 0.0011112689971923828 nb_pixel_total : 23476 time to create 1 rle with old method : 0.025051116943359375 length of segment : 198 time for calcul the mask position with numpy : 0.0021369457244873047 nb_pixel_total : 56059 time to create 1 rle with old method : 0.05937671661376953 length of segment : 317 time for calcul the mask position with numpy : 0.0011820793151855469 nb_pixel_total : 25092 time to create 1 rle with old method : 0.02772355079650879 length of segment : 199 time for calcul the mask position with numpy : 0.005201101303100586 nb_pixel_total : 140656 time to create 1 rle with old method : 0.15593290328979492 length of segment : 773 time for calcul the mask position with numpy : 0.0013778209686279297 nb_pixel_total : 27713 time to create 1 rle with old method : 0.031207561492919922 length of segment : 297 time for calcul the mask position with numpy : 0.0011663436889648438 nb_pixel_total : 23346 time to create 1 rle with old method : 0.025936126708984375 length of segment : 288 time for calcul the mask position with numpy : 0.0018405914306640625 nb_pixel_total : 36898 time to create 1 rle with old method : 0.04003119468688965 length of segment : 304 time for calcul the mask position with numpy : 0.0007448196411132812 nb_pixel_total : 19912 time to create 1 rle with old method : 0.022315502166748047 length of segment : 209 time for calcul the mask position with numpy : 0.0007266998291015625 nb_pixel_total : 15537 time to create 1 rle with old method : 0.01755833625793457 length of segment : 208 time for calcul the mask position with numpy : 0.0014603137969970703 nb_pixel_total : 26271 time to create 1 rle with old method : 0.02922987937927246 length of segment : 172 time for calcul the mask position with numpy : 0.0029180049896240234 nb_pixel_total : 69917 time to create 1 rle with old method : 0.07541155815124512 length of segment : 346 time for calcul the mask position with numpy : 0.0008516311645507812 nb_pixel_total : 19135 time to create 1 rle with old method : 0.021144866943359375 length of segment : 139 time for calcul the mask position with numpy : 0.002277374267578125 nb_pixel_total : 44872 time to create 1 rle with old method : 0.04815673828125 length of segment : 253 time for calcul the mask position with numpy : 0.0018229484558105469 nb_pixel_total : 50823 time to create 1 rle with old method : 0.05619406700134277 length of segment : 292 time for calcul the mask position with numpy : 0.002579927444458008 nb_pixel_total : 73329 time to create 1 rle with old method : 0.08333921432495117 length of segment : 471 time for calcul the mask position with numpy : 0.0018033981323242188 nb_pixel_total : 29781 time to create 1 rle with old method : 0.033480167388916016 length of segment : 269 time for calcul the mask position with numpy : 0.002635955810546875 nb_pixel_total : 52106 time to create 1 rle with old method : 0.06501460075378418 length of segment : 260 time for calcul the mask position with numpy : 0.001439809799194336 nb_pixel_total : 37662 time to create 1 rle with old method : 0.04185032844543457 length of segment : 236 time for calcul the mask position with numpy : 0.0013585090637207031 nb_pixel_total : 28380 time to create 1 rle with old method : 0.034795522689819336 length of segment : 196 time for calcul the mask position with numpy : 0.0016605854034423828 nb_pixel_total : 38521 time to create 1 rle with old method : 0.04245162010192871 length of segment : 260 time for calcul the mask position with numpy : 0.0009529590606689453 nb_pixel_total : 22734 time to create 1 rle with old method : 0.024697065353393555 length of segment : 198 time for calcul the mask position with numpy : 0.0012860298156738281 nb_pixel_total : 24820 time to create 1 rle with old method : 0.026540279388427734 length of segment : 485 time for calcul the mask position with numpy : 0.001116037368774414 nb_pixel_total : 27863 time to create 1 rle with old method : 0.03005671501159668 length of segment : 179 time for calcul the mask position with numpy : 0.00024008750915527344 nb_pixel_total : 5806 time to create 1 rle with old method : 0.006766557693481445 length of segment : 111 time for calcul the mask position with numpy : 0.015168428421020508 nb_pixel_total : 528171 time to create 1 rle with new method : 0.028239965438842773 length of segment : 927 time for calcul the mask position with numpy : 0.00018453598022460938 nb_pixel_total : 8560 time to create 1 rle with old method : 0.009975910186767578 length of segment : 116 time for calcul the mask position with numpy : 0.0016644001007080078 nb_pixel_total : 62049 time to create 1 rle with old method : 0.06509566307067871 length of segment : 341 time for calcul the mask position with numpy : 0.0007200241088867188 nb_pixel_total : 35877 time to create 1 rle with old method : 0.038286447525024414 length of segment : 374 time for calcul the mask position with numpy : 0.0007684230804443359 nb_pixel_total : 24149 time to create 1 rle with old method : 0.026508808135986328 length of segment : 173 time for calcul the mask position with numpy : 0.0007822513580322266 nb_pixel_total : 23213 time to create 1 rle with old method : 0.02617955207824707 length of segment : 267 time for calcul the mask position with numpy : 0.0005538463592529297 nb_pixel_total : 14818 time to create 1 rle with old method : 0.01708841323852539 length of segment : 155 time for calcul the mask position with numpy : 0.0019006729125976562 nb_pixel_total : 67292 time to create 1 rle with old method : 0.07592511177062988 length of segment : 366 time for calcul the mask position with numpy : 0.00034928321838378906 nb_pixel_total : 14203 time to create 1 rle with old method : 0.01564931869506836 length of segment : 167 time for calcul the mask position with numpy : 0.001153707504272461 nb_pixel_total : 50023 time to create 1 rle with old method : 0.053763628005981445 length of segment : 266 time for calcul the mask position with numpy : 0.0007815361022949219 nb_pixel_total : 25512 time to create 1 rle with old method : 0.02834939956665039 length of segment : 183 time for calcul the mask position with numpy : 0.0006787776947021484 nb_pixel_total : 26691 time to create 1 rle with old method : 0.029112815856933594 length of segment : 186 time for calcul the mask position with numpy : 0.00047469139099121094 nb_pixel_total : 19373 time to create 1 rle with old method : 0.022548437118530273 length of segment : 173 time for calcul the mask position with numpy : 0.0008380413055419922 nb_pixel_total : 17826 time to create 1 rle with old method : 0.020551443099975586 length of segment : 210 time for calcul the mask position with numpy : 0.0010149478912353516 nb_pixel_total : 42624 time to create 1 rle with old method : 0.04798579216003418 length of segment : 276 time for calcul the mask position with numpy : 0.0003724098205566406 nb_pixel_total : 13835 time to create 1 rle with old method : 0.016112089157104492 length of segment : 151 time for calcul the mask position with numpy : 0.0004763603210449219 nb_pixel_total : 16108 time to create 1 rle with old method : 0.018143892288208008 length of segment : 191 time for calcul the mask position with numpy : 0.0019390583038330078 nb_pixel_total : 63136 time to create 1 rle with old method : 0.06592488288879395 length of segment : 439 time for calcul the mask position with numpy : 0.0009653568267822266 nb_pixel_total : 19758 time to create 1 rle with old method : 0.021810531616210938 length of segment : 184 time for calcul the mask position with numpy : 0.0030236244201660156 nb_pixel_total : 78316 time to create 1 rle with old method : 0.0856025218963623 length of segment : 431 time for calcul the mask position with numpy : 0.0015919208526611328 nb_pixel_total : 42221 time to create 1 rle with old method : 0.04731106758117676 length of segment : 291 time for calcul the mask position with numpy : 0.003871917724609375 nb_pixel_total : 138152 time to create 1 rle with old method : 0.142289400100708 length of segment : 485 time for calcul the mask position with numpy : 0.0004642009735107422 nb_pixel_total : 13195 time to create 1 rle with old method : 0.014882564544677734 length of segment : 148 time for calcul the mask position with numpy : 0.0026602745056152344 nb_pixel_total : 84579 time to create 1 rle with old method : 0.08935809135437012 length of segment : 453 time for calcul the mask position with numpy : 0.0015087127685546875 nb_pixel_total : 43651 time to create 1 rle with old method : 0.046591758728027344 length of segment : 291 time for calcul the mask position with numpy : 0.00031828880310058594 nb_pixel_total : 5579 time to create 1 rle with old method : 0.0066378116607666016 length of segment : 81 time for calcul the mask position with numpy : 0.0005393028259277344 nb_pixel_total : 17806 time to create 1 rle with old method : 0.01879596710205078 length of segment : 164 time for calcul the mask position with numpy : 0.0004291534423828125 nb_pixel_total : 14468 time to create 1 rle with old method : 0.015770673751831055 length of segment : 124 time for calcul the mask position with numpy : 0.008058786392211914 nb_pixel_total : 113371 time to create 1 rle with old method : 0.11671257019042969 length of segment : 491 time for calcul the mask position with numpy : 0.0012583732604980469 nb_pixel_total : 31542 time to create 1 rle with old method : 0.03247666358947754 length of segment : 307 time for calcul the mask position with numpy : 0.0008769035339355469 nb_pixel_total : 22350 time to create 1 rle with old method : 0.023035287857055664 length of segment : 273 time for calcul the mask position with numpy : 0.0007452964782714844 nb_pixel_total : 20290 time to create 1 rle with old method : 0.022159099578857422 length of segment : 167 time for calcul the mask position with numpy : 0.0009317398071289062 nb_pixel_total : 20382 time to create 1 rle with old method : 0.03330802917480469 length of segment : 240 time for calcul the mask position with numpy : 0.0003962516784667969 nb_pixel_total : 24408 time to create 1 rle with old method : 0.027700424194335938 length of segment : 156 time for calcul the mask position with numpy : 0.0008406639099121094 nb_pixel_total : 15915 time to create 1 rle with old method : 0.01825237274169922 length of segment : 340 time for calcul the mask position with numpy : 0.0006721019744873047 nb_pixel_total : 16118 time to create 1 rle with old method : 0.018148422241210938 length of segment : 121 time for calcul the mask position with numpy : 0.00015354156494140625 nb_pixel_total : 5654 time to create 1 rle with old method : 0.00673365592956543 length of segment : 137 time for calcul the mask position with numpy : 0.0010581016540527344 nb_pixel_total : 66022 time to create 1 rle with old method : 0.07165789604187012 length of segment : 206 time for calcul the mask position with numpy : 0.0015516281127929688 nb_pixel_total : 83987 time to create 1 rle with old method : 0.0881810188293457 length of segment : 402 time for calcul the mask position with numpy : 0.00030040740966796875 nb_pixel_total : 8911 time to create 1 rle with old method : 0.010447263717651367 length of segment : 170 time for calcul the mask position with numpy : 0.0005118846893310547 nb_pixel_total : 20207 time to create 1 rle with old method : 0.023076295852661133 length of segment : 199 time for calcul the mask position with numpy : 0.0004892349243164062 nb_pixel_total : 19563 time to create 1 rle with old method : 0.02162909507751465 length of segment : 227 time for calcul the mask position with numpy : 0.0012180805206298828 nb_pixel_total : 32181 time to create 1 rle with old method : 0.0380854606628418 length of segment : 257 time for calcul the mask position with numpy : 0.0011947154998779297 nb_pixel_total : 38077 time to create 1 rle with old method : 0.042760610580444336 length of segment : 329 time for calcul the mask position with numpy : 0.0033559799194335938 nb_pixel_total : 95553 time to create 1 rle with old method : 0.13124632835388184 length of segment : 371 time for calcul the mask position with numpy : 0.0011675357818603516 nb_pixel_total : 60559 time to create 1 rle with old method : 0.0669102668762207 length of segment : 255 time for calcul the mask position with numpy : 0.0004069805145263672 nb_pixel_total : 11653 time to create 1 rle with old method : 0.013049602508544922 length of segment : 186 time for calcul the mask position with numpy : 0.0003142356872558594 nb_pixel_total : 16053 time to create 1 rle with old method : 0.017951488494873047 length of segment : 144 time for calcul the mask position with numpy : 0.0005276203155517578 nb_pixel_total : 24531 time to create 1 rle with old method : 0.026484966278076172 length of segment : 315 time for calcul the mask position with numpy : 0.0003726482391357422 nb_pixel_total : 10912 time to create 1 rle with old method : 0.012473821640014648 length of segment : 138 time for calcul the mask position with numpy : 0.00048232078552246094 nb_pixel_total : 16480 time to create 1 rle with old method : 0.018061399459838867 length of segment : 148 time for calcul the mask position with numpy : 0.00041866302490234375 nb_pixel_total : 21675 time to create 1 rle with old method : 0.023777008056640625 length of segment : 247 time for calcul the mask position with numpy : 0.0022253990173339844 nb_pixel_total : 122786 time to create 1 rle with old method : 0.12987804412841797 length of segment : 537 time for calcul the mask position with numpy : 0.0009925365447998047 nb_pixel_total : 39568 time to create 1 rle with old method : 0.04323077201843262 length of segment : 302 time for calcul the mask position with numpy : 0.0002522468566894531 nb_pixel_total : 9771 time to create 1 rle with old method : 0.011168956756591797 length of segment : 247 time for calcul the mask position with numpy : 0.0017588138580322266 nb_pixel_total : 82902 time to create 1 rle with old method : 0.08790397644042969 length of segment : 532 time for calcul the mask position with numpy : 0.000993490219116211 nb_pixel_total : 58113 time to create 1 rle with old method : 0.0629422664642334 length of segment : 219 time for calcul the mask position with numpy : 0.0009417533874511719 nb_pixel_total : 46421 time to create 1 rle with old method : 0.05122637748718262 length of segment : 190 time for calcul the mask position with numpy : 0.0008952617645263672 nb_pixel_total : 42155 time to create 1 rle with old method : 0.04648232460021973 length of segment : 233 time for calcul the mask position with numpy : 0.0008318424224853516 nb_pixel_total : 22870 time to create 1 rle with old method : 0.02657628059387207 length of segment : 232 time for calcul the mask position with numpy : 0.0010149478912353516 nb_pixel_total : 34359 time to create 1 rle with old method : 0.0381317138671875 length of segment : 294 time for calcul the mask position with numpy : 0.0013191699981689453 nb_pixel_total : 70176 time to create 1 rle with old method : 0.0739283561706543 length of segment : 272 time for calcul the mask position with numpy : 0.0019712448120117188 nb_pixel_total : 58682 time to create 1 rle with old method : 0.06281542778015137 length of segment : 448 time for calcul the mask position with numpy : 0.0007302761077880859 nb_pixel_total : 32687 time to create 1 rle with old method : 0.03501319885253906 length of segment : 220 time for calcul the mask position with numpy : 0.0075986385345458984 nb_pixel_total : 261066 time to create 1 rle with new method : 0.02033233642578125 length of segment : 912 time for calcul the mask position with numpy : 0.00045800209045410156 nb_pixel_total : 17518 time to create 1 rle with old method : 0.01943349838256836 length of segment : 188 time for calcul the mask position with numpy : 0.0003886222839355469 nb_pixel_total : 14039 time to create 1 rle with old method : 0.01567530632019043 length of segment : 208 time for calcul the mask position with numpy : 0.0009751319885253906 nb_pixel_total : 38648 time to create 1 rle with old method : 0.04263734817504883 length of segment : 193 time for calcul the mask position with numpy : 0.0006511211395263672 nb_pixel_total : 20066 time to create 1 rle with old method : 0.02122664451599121 length of segment : 244 time for calcul the mask position with numpy : 0.0034165382385253906 nb_pixel_total : 144405 time to create 1 rle with old method : 0.15923762321472168 length of segment : 654 time for calcul the mask position with numpy : 0.0015730857849121094 nb_pixel_total : 19681 time to create 1 rle with old method : 0.021239280700683594 length of segment : 712 time for calcul the mask position with numpy : 0.00048351287841796875 nb_pixel_total : 21658 time to create 1 rle with old method : 0.02448892593383789 length of segment : 162 time for calcul the mask position with numpy : 0.0014050006866455078 nb_pixel_total : 12213 time to create 1 rle with old method : 0.014023303985595703 length of segment : 300 time for calcul the mask position with numpy : 0.0016498565673828125 nb_pixel_total : 34627 time to create 1 rle with old method : 0.03668093681335449 length of segment : 321 time for calcul the mask position with numpy : 0.0006160736083984375 nb_pixel_total : 16679 time to create 1 rle with old method : 0.018012285232543945 length of segment : 140 time for calcul the mask position with numpy : 0.0007159709930419922 nb_pixel_total : 15821 time to create 1 rle with old method : 0.01903367042541504 length of segment : 163 time spent for convertir_results : 12.957794427871704 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 160 chid ids of type : 3594 Number RLEs to save : 45353 save missing photos in datou_result : time spend for datou_step_exec : 94.44476079940796 time spend to save output : 2.8130242824554443 total time spend for step 1 : 97.2577850818634 step2:crop_condition Mon Feb 10 16:42:07 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 8 ! batch 1 Loaded 160 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 124 About to insert : list_path_to_insert length 124 new photo from crops ! About to upload 124 photos upload in portfolio : 3736932 init cache_photo without model_param we have 124 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739202159_3153736 we have uploaded 124 photos in the portfolio 3736932 time of upload the photos Elapsed time : 30.429816246032715 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 25 About to insert : list_path_to_insert length 25 new photo from crops ! About to upload 25 photos upload in portfolio : 3736932 init cache_photo without model_param we have 25 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739202194_3153736 we have uploaded 25 photos in the portfolio 3736932 time of upload the photos Elapsed time : 7.344452142715454 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3736932 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739202203_3153736 we have uploaded 2 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.0285885334014893 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! About to upload 6 photos upload in portfolio : 3736932 init cache_photo without model_param we have 6 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739202207_3153736 we have uploaded 6 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.72113037109375 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3736932 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739202210_3153736 we have uploaded 2 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.7798116207122803 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739202212_3153736 we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.5537779331207275 we have finished the crop for the class : pehd begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1336528979, 1336528884, 1336528786, 1336528724, 1336528189, 1336528104, 1336528004, 1336527904] Looping around the photos to save general results len do output : 160 /1336536099Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536102Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536103Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536104Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536105Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536106Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536107Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536108Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536109Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536110Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536111Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536112Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536113Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536114Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536115Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536116Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536117Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536118Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536119Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536120Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536121Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536122Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536123Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536124Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536125Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536126Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536127Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536128Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536129Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536130Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536131Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536132Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536133Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536134Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536135Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536136Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536137Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536138Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536139Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536140Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536141Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536142Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536143Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536144Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536145Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536146Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536147Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536148Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536149Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536150Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536151Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536152Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536153Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536154Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536155Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536156Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536157Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536158Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536159Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536160Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536161Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536162Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536163Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536164Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536165Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536166Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536167Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536168Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536170Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536171Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536172Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536173Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536174Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536175Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536176Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536177Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536178Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536179Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536180Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536181Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536182Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536183Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536184Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536185Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536186Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536187Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536188Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536189Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536190Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536191Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536192Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536193Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536194Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536195Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536196Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536197Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536198Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536199Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536200Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536201Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536202Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536203Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536204Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536205Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536206Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536207Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536208Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536209Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536210Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536212Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536213Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536214Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536215Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536216Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536217Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536218Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536219Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536220Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536221Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536222Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536223Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536224Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536225Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536226Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536235Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536236Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536237Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536238Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536239Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536240Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536241Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536242Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536243Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536244Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536245Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536246Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536248Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536249Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536250Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536251Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536252Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536254Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536255Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536256Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536257Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536258Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536259Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536260Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536261Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536262Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536263Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536266Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536267Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536268Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536269Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536271Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536272Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536275Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536276Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336536277Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528979', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528884', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528786', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528724', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528189', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528104', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528004', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336527904', None, None, None, None, None, '2571901') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 488 time used for this insertion : 0.02962636947631836 save_final save missing photos in datou_result : time spend for datou_step_exec : 85.08193063735962 time spend to save output : 0.03406667709350586 total time spend for step 2 : 85.11599731445312 step3:rle_unique_nms_with_priority Mon Feb 10 16:43:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 160 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 19 nb_hashtags : 4 time to prepare the origin masks : 6.5586607456207275 time for calcul the mask position with numpy : 0.533635139465332 nb_pixel_total : 7460167 time to create 1 rle with new method : 0.8011837005615234 time for calcul the mask position with numpy : 0.024206161499023438 nb_pixel_total : 20278 time to create 1 rle with old method : 0.023047208786010742 time for calcul the mask position with numpy : 0.027679920196533203 nb_pixel_total : 38335 time to create 1 rle with old method : 0.04290604591369629 time for calcul the mask position with numpy : 0.02613210678100586 nb_pixel_total : 42782 time to create 1 rle with old method : 0.04630899429321289 time for calcul the mask position with numpy : 0.025264263153076172 nb_pixel_total : 30202 time to create 1 rle with old method : 0.03213191032409668 time for calcul the mask position with numpy : 0.025311708450317383 nb_pixel_total : 10224 time to create 1 rle with old method : 0.011438608169555664 time for calcul the mask position with numpy : 0.03231668472290039 nb_pixel_total : 22143 time to create 1 rle with old method : 0.026608943939208984 time for calcul the mask position with numpy : 0.025376081466674805 nb_pixel_total : 44825 time to create 1 rle with old method : 0.04904770851135254 time for calcul the mask position with numpy : 0.025936603546142578 nb_pixel_total : 52839 time to create 1 rle with old method : 0.07869625091552734 time for calcul the mask position with numpy : 0.024876117706298828 nb_pixel_total : 67689 time to create 1 rle with old method : 0.07209181785583496 time for calcul the mask position with numpy : 0.025850534439086914 nb_pixel_total : 75136 time to create 1 rle with old method : 0.0807645320892334 time for calcul the mask position with numpy : 0.025919675827026367 nb_pixel_total : 35304 time to create 1 rle with old method : 0.03697776794433594 time for calcul the mask position with numpy : 0.02447032928466797 nb_pixel_total : 9603 time to create 1 rle with old method : 0.01162409782409668 time for calcul the mask position with numpy : 0.024669885635375977 nb_pixel_total : 37485 time to create 1 rle with old method : 0.040447235107421875 time for calcul the mask position with numpy : 0.02671980857849121 nb_pixel_total : 79503 time to create 1 rle with old method : 0.08598208427429199 time for calcul the mask position with numpy : 0.02453136444091797 nb_pixel_total : 36362 time to create 1 rle with old method : 0.03833174705505371 time for calcul the mask position with numpy : 0.02532792091369629 nb_pixel_total : 33264 time to create 1 rle with old method : 0.036577463150024414 time for calcul the mask position with numpy : 0.027425050735473633 nb_pixel_total : 65301 time to create 1 rle with old method : 0.09027457237243652 time for calcul the mask position with numpy : 0.03188014030456543 nb_pixel_total : 101689 time to create 1 rle with old method : 0.10968303680419922 time for calcul the mask position with numpy : 0.024714946746826172 nb_pixel_total : 31269 time to create 1 rle with old method : 0.034233808517456055 create new chi : 2.820957899093628 time to delete rle : 0.014863014221191406 batch 1 Loaded 39 chid ids of type : 3594 ++++++++++++++++++++++++++Number RLEs to save : 12763 TO DO : save crop sub photo not yet done ! save time : 0.7593438625335693 nb_obj : 17 nb_hashtags : 3 time to prepare the origin masks : 5.9664812088012695 time for calcul the mask position with numpy : 0.44135570526123047 nb_pixel_total : 7497731 time to create 1 rle with new method : 0.9976704120635986 time for calcul the mask position with numpy : 0.028847694396972656 nb_pixel_total : 90105 time to create 1 rle with old method : 0.09588384628295898 time for calcul the mask position with numpy : 0.04109501838684082 nb_pixel_total : 7594 time to create 1 rle with old method : 0.008496522903442383 time for calcul the mask position with numpy : 0.041207075119018555 nb_pixel_total : 113045 time to create 1 rle with old method : 0.11881113052368164 time for calcul the mask position with numpy : 0.0382685661315918 nb_pixel_total : 122418 time to create 1 rle with old method : 0.14699006080627441 time for calcul the mask position with numpy : 0.03943896293640137 nb_pixel_total : 63495 time to create 1 rle with old method : 0.06737542152404785 time for calcul the mask position with numpy : 0.024722576141357422 nb_pixel_total : 25282 time to create 1 rle with old method : 0.02842569351196289 time for calcul the mask position with numpy : 0.025616884231567383 nb_pixel_total : 15694 time to create 1 rle with old method : 0.017194509506225586 time for calcul the mask position with numpy : 0.026324033737182617 nb_pixel_total : 16874 time to create 1 rle with old method : 0.01794886589050293 time for calcul the mask position with numpy : 0.025331974029541016 nb_pixel_total : 31285 time to create 1 rle with old method : 0.03390669822692871 time for calcul the mask position with numpy : 0.025256633758544922 nb_pixel_total : 36307 time to create 1 rle with old method : 0.038825035095214844 time for calcul the mask position with numpy : 0.025033950805664062 nb_pixel_total : 27807 time to create 1 rle with old method : 0.029703140258789062 time for calcul the mask position with numpy : 0.02426767349243164 nb_pixel_total : 26128 time to create 1 rle with old method : 0.0286562442779541 time for calcul the mask position with numpy : 0.025621414184570312 nb_pixel_total : 31221 time to create 1 rle with old method : 0.03290438652038574 time for calcul the mask position with numpy : 0.024389266967773438 nb_pixel_total : 76301 time to create 1 rle with old method : 0.0809023380279541 time for calcul the mask position with numpy : 0.026023387908935547 nb_pixel_total : 80863 time to create 1 rle with old method : 0.08433103561401367 time for calcul the mask position with numpy : 0.030147314071655273 nb_pixel_total : 22513 time to create 1 rle with old method : 0.024129867553710938 time for calcul the mask position with numpy : 0.04010510444641113 nb_pixel_total : 9737 time to create 1 rle with old method : 0.01038050651550293 create new chi : 2.8629941940307617 time to delete rle : 0.0017559528350830078 batch 1 Loaded 35 chid ids of type : 3594 ++++++++++++++++++++++++++Number RLEs to save : 12550 TO DO : save crop sub photo not yet done ! save time : 1.9805030822753906 nb_obj : 18 nb_hashtags : 3 time to prepare the origin masks : 5.987276792526245 time for calcul the mask position with numpy : 0.9262831211090088 nb_pixel_total : 7205807 time to create 1 rle with new method : 0.48905420303344727 time for calcul the mask position with numpy : 0.025358200073242188 nb_pixel_total : 17647 time to create 1 rle with old method : 0.019514083862304688 time for calcul the mask position with numpy : 0.0411381721496582 nb_pixel_total : 9644 time to create 1 rle with old method : 0.011756181716918945 time for calcul the mask position with numpy : 0.02585291862487793 nb_pixel_total : 19357 time to create 1 rle with old method : 0.022207260131835938 time for calcul the mask position with numpy : 0.026036739349365234 nb_pixel_total : 13137 time to create 1 rle with old method : 0.014739990234375 time for calcul the mask position with numpy : 0.027926921844482422 nb_pixel_total : 54740 time to create 1 rle with old method : 0.05941939353942871 time for calcul the mask position with numpy : 0.024968862533569336 nb_pixel_total : 22187 time to create 1 rle with old method : 0.023818016052246094 time for calcul the mask position with numpy : 0.026378154754638672 nb_pixel_total : 155908 time to create 1 rle with new method : 0.60296630859375 time for calcul the mask position with numpy : 0.029073476791381836 nb_pixel_total : 427062 time to create 1 rle with new method : 0.5299746990203857 time for calcul the mask position with numpy : 0.02416539192199707 nb_pixel_total : 9817 time to create 1 rle with old method : 0.010788202285766602 time for calcul the mask position with numpy : 0.024293184280395508 nb_pixel_total : 58480 time to create 1 rle with old method : 0.06438589096069336 time for calcul the mask position with numpy : 0.025795936584472656 nb_pixel_total : 71593 time to create 1 rle with old method : 0.08204936981201172 time for calcul the mask position with numpy : 0.024886131286621094 nb_pixel_total : 46948 time to create 1 rle with old method : 0.051361799240112305 time for calcul the mask position with numpy : 0.027992963790893555 nb_pixel_total : 42178 time to create 1 rle with old method : 0.04661369323730469 time for calcul the mask position with numpy : 0.02931380271911621 nb_pixel_total : 13452 time to create 1 rle with old method : 0.015139102935791016 time for calcul the mask position with numpy : 0.02826237678527832 nb_pixel_total : 35849 time to create 1 rle with old method : 0.039337873458862305 time for calcul the mask position with numpy : 0.03969740867614746 nb_pixel_total : 38761 time to create 1 rle with old method : 0.040740013122558594 time for calcul the mask position with numpy : 0.039096832275390625 nb_pixel_total : 15078 time to create 1 rle with old method : 0.016150474548339844 time for calcul the mask position with numpy : 0.03912353515625 nb_pixel_total : 36755 time to create 1 rle with old method : 0.03886914253234863 create new chi : 3.73661470413208 time to delete rle : 0.001847982406616211 batch 1 Loaded 37 chid ids of type : 3594 +++++++++++++++++++++++++Number RLEs to save : 12917 TO DO : save crop sub photo not yet done ! save time : 1.5817947387695312 nb_obj : 21 nb_hashtags : 2 time to prepare the origin masks : 6.287526607513428 time for calcul the mask position with numpy : 0.3922905921936035 nb_pixel_total : 7511374 time to create 1 rle with new method : 0.6095035076141357 time for calcul the mask position with numpy : 0.026404380798339844 nb_pixel_total : 73329 time to create 1 rle with old method : 0.07810378074645996 time for calcul the mask position with numpy : 0.03978538513183594 nb_pixel_total : 50823 time to create 1 rle with old method : 0.05431818962097168 time for calcul the mask position with numpy : 0.04009509086608887 nb_pixel_total : 44872 time to create 1 rle with old method : 0.04669618606567383 time for calcul the mask position with numpy : 0.023807764053344727 nb_pixel_total : 19135 time to create 1 rle with old method : 0.020128250122070312 time for calcul the mask position with numpy : 0.024254798889160156 nb_pixel_total : 69917 time to create 1 rle with old method : 0.07387423515319824 time for calcul the mask position with numpy : 0.024028539657592773 nb_pixel_total : 26271 time to create 1 rle with old method : 0.02860403060913086 time for calcul the mask position with numpy : 0.025774478912353516 nb_pixel_total : 15537 time to create 1 rle with old method : 0.01658153533935547 time for calcul the mask position with numpy : 0.02482891082763672 nb_pixel_total : 19912 time to create 1 rle with old method : 0.02125096321105957 time for calcul the mask position with numpy : 0.02390289306640625 nb_pixel_total : 36898 time to create 1 rle with old method : 0.03980517387390137 time for calcul the mask position with numpy : 0.02656698226928711 nb_pixel_total : 23346 time to create 1 rle with old method : 0.037911176681518555 time for calcul the mask position with numpy : 0.027958154678344727 nb_pixel_total : 27713 time to create 1 rle with old method : 0.03253054618835449 time for calcul the mask position with numpy : 0.024691343307495117 nb_pixel_total : 124204 time to create 1 rle with old method : 0.1319565773010254 time for calcul the mask position with numpy : 0.0251004695892334 nb_pixel_total : 25092 time to create 1 rle with old method : 0.026961565017700195 time for calcul the mask position with numpy : 0.0260162353515625 nb_pixel_total : 56059 time to create 1 rle with old method : 0.060053110122680664 time for calcul the mask position with numpy : 0.025374889373779297 nb_pixel_total : 23476 time to create 1 rle with old method : 0.026074886322021484 time for calcul the mask position with numpy : 0.025650501251220703 nb_pixel_total : 25357 time to create 1 rle with old method : 0.030124425888061523 time for calcul the mask position with numpy : 0.025014638900756836 nb_pixel_total : 17050 time to create 1 rle with old method : 0.01807999610900879 time for calcul the mask position with numpy : 0.024022817611694336 nb_pixel_total : 15669 time to create 1 rle with old method : 0.016630172729492188 time for calcul the mask position with numpy : 0.023961782455444336 nb_pixel_total : 32520 time to create 1 rle with old method : 0.03433632850646973 time for calcul the mask position with numpy : 0.024168014526367188 nb_pixel_total : 17378 time to create 1 rle with old method : 0.01833796501159668 time for calcul the mask position with numpy : 0.02411818504333496 nb_pixel_total : 38468 time to create 1 rle with old method : 0.04068470001220703 create new chi : 2.4554333686828613 time to delete rle : 0.0016965866088867188 batch 1 Loaded 43 chid ids of type : 3594 +++++++++++++++++++++++Number RLEs to save : 13077 TO DO : save crop sub photo not yet done ! save time : 0.777531623840332 nb_obj : 26 nb_hashtags : 3 time to prepare the origin masks : 4.403570890426636 time for calcul the mask position with numpy : 0.08492231369018555 nb_pixel_total : 7052938 time to create 1 rle with new method : 0.7572000026702881 time for calcul the mask position with numpy : 0.034325599670410156 nb_pixel_total : 52106 time to create 1 rle with old method : 0.059235334396362305 time for calcul the mask position with numpy : 0.0341646671295166 nb_pixel_total : 25512 time to create 1 rle with old method : 0.028098106384277344 time for calcul the mask position with numpy : 0.03415870666503906 nb_pixel_total : 24766 time to create 1 rle with old method : 0.027818679809570312 time for calcul the mask position with numpy : 0.03359723091125488 nb_pixel_total : 24820 time to create 1 rle with old method : 0.02752971649169922 time for calcul the mask position with numpy : 0.03280282020568848 nb_pixel_total : 50023 time to create 1 rle with old method : 0.05413484573364258 time for calcul the mask position with numpy : 0.03373575210571289 nb_pixel_total : 37662 time to create 1 rle with old method : 0.04061746597290039 time for calcul the mask position with numpy : 0.03330230712890625 nb_pixel_total : 38521 time to create 1 rle with old method : 0.03983712196350098 time for calcul the mask position with numpy : 0.03142142295837402 nb_pixel_total : 29781 time to create 1 rle with old method : 0.03174328804016113 time for calcul the mask position with numpy : 0.0324552059173584 nb_pixel_total : 24149 time to create 1 rle with old method : 0.024461030960083008 time for calcul the mask position with numpy : 0.031702518463134766 nb_pixel_total : 19373 time to create 1 rle with old method : 0.020893096923828125 time for calcul the mask position with numpy : 0.033612966537475586 nb_pixel_total : 16108 time to create 1 rle with old method : 0.01718735694885254 time for calcul the mask position with numpy : 0.03220009803771973 nb_pixel_total : 17733 time to create 1 rle with old method : 0.018978357315063477 time for calcul the mask position with numpy : 0.03280758857727051 nb_pixel_total : 67292 time to create 1 rle with old method : 0.06961464881896973 time for calcul the mask position with numpy : 0.03373122215270996 nb_pixel_total : 62049 time to create 1 rle with old method : 0.06530141830444336 time for calcul the mask position with numpy : 0.032446861267089844 nb_pixel_total : 23213 time to create 1 rle with old method : 0.023492097854614258 time for calcul the mask position with numpy : 0.03185701370239258 nb_pixel_total : 27863 time to create 1 rle with old method : 0.0289614200592041 time for calcul the mask position with numpy : 0.034777164459228516 nb_pixel_total : 522365 time to create 1 rle with new method : 0.14452743530273438 time for calcul the mask position with numpy : 0.031633853912353516 nb_pixel_total : 42624 time to create 1 rle with old method : 0.042818307876586914 time for calcul the mask position with numpy : 0.03228497505187988 nb_pixel_total : 22734 time to create 1 rle with old method : 0.023521900177001953 time for calcul the mask position with numpy : 0.032057762145996094 nb_pixel_total : 14818 time to create 1 rle with old method : 0.015333175659179688 time for calcul the mask position with numpy : 0.03139805793762207 nb_pixel_total : 14203 time to create 1 rle with old method : 0.014622926712036133 time for calcul the mask position with numpy : 0.031072616577148438 nb_pixel_total : 35691 time to create 1 rle with old method : 0.035834550857543945 time for calcul the mask position with numpy : 0.03125357627868652 nb_pixel_total : 35 time to create 1 rle with old method : 6.103515625e-05 time for calcul the mask position with numpy : 0.030846357345581055 nb_pixel_total : 5806 time to create 1 rle with old method : 0.006034374237060547 time for calcul the mask position with numpy : 0.031290292739868164 nb_pixel_total : 28380 time to create 1 rle with old method : 0.02909398078918457 time for calcul the mask position with numpy : 0.031208515167236328 nb_pixel_total : 13835 time to create 1 rle with old method : 0.013707399368286133 create new chi : 2.6323909759521484 time to delete rle : 0.0021173954010009766 batch 1 Loaded 53 chid ids of type : 3594 +++++++++++++++++++++++++++++++Number RLEs to save : 15316 TO DO : save crop sub photo not yet done ! save time : 0.8909649848937988 nb_obj : 18 nb_hashtags : 2 time to prepare the origin masks : 6.377596139907837 time for calcul the mask position with numpy : 0.6873998641967773 nb_pixel_total : 7539172 time to create 1 rle with new method : 0.7072489261627197 time for calcul the mask position with numpy : 0.02426910400390625 nb_pixel_total : 16118 time to create 1 rle with old method : 0.018327713012695312 time for calcul the mask position with numpy : 0.02391672134399414 nb_pixel_total : 15915 time to create 1 rle with old method : 0.017916440963745117 time for calcul the mask position with numpy : 0.02460479736328125 nb_pixel_total : 20382 time to create 1 rle with old method : 0.022312164306640625 time for calcul the mask position with numpy : 0.024814367294311523 nb_pixel_total : 20290 time to create 1 rle with old method : 0.023087501525878906 time for calcul the mask position with numpy : 0.02542734146118164 nb_pixel_total : 22350 time to create 1 rle with old method : 0.024370193481445312 time for calcul the mask position with numpy : 0.027031898498535156 nb_pixel_total : 31542 time to create 1 rle with old method : 0.034301042556762695 time for calcul the mask position with numpy : 0.03274798393249512 nb_pixel_total : 107770 time to create 1 rle with old method : 0.1156930923461914 time for calcul the mask position with numpy : 0.04161524772644043 nb_pixel_total : 14468 time to create 1 rle with old method : 0.015644550323486328 time for calcul the mask position with numpy : 0.03961181640625 nb_pixel_total : 17806 time to create 1 rle with old method : 0.01889801025390625 time for calcul the mask position with numpy : 0.039835453033447266 nb_pixel_total : 5579 time to create 1 rle with old method : 0.006272315979003906 time for calcul the mask position with numpy : 0.03651714324951172 nb_pixel_total : 43651 time to create 1 rle with old method : 0.04648423194885254 time for calcul the mask position with numpy : 0.02655768394470215 nb_pixel_total : 84579 time to create 1 rle with old method : 0.08995795249938965 time for calcul the mask position with numpy : 0.024390459060668945 nb_pixel_total : 13195 time to create 1 rle with old method : 0.014029264450073242 time for calcul the mask position with numpy : 0.024754762649536133 nb_pixel_total : 138152 time to create 1 rle with old method : 0.14609122276306152 time for calcul the mask position with numpy : 0.023668766021728516 nb_pixel_total : 42221 time to create 1 rle with old method : 0.04553413391113281 time for calcul the mask position with numpy : 0.024898529052734375 nb_pixel_total : 78316 time to create 1 rle with old method : 0.0995333194732666 time for calcul the mask position with numpy : 0.027286767959594727 nb_pixel_total : 19758 time to create 1 rle with old method : 0.023866891860961914 time for calcul the mask position with numpy : 0.025165319442749023 nb_pixel_total : 63136 time to create 1 rle with old method : 0.06878495216369629 create new chi : 2.783045768737793 time to delete rle : 0.002054452896118164 batch 1 Loaded 39 chid ids of type : 3594 +++++++++++++++++++++++++++++++Number RLEs to save : 12214 TO DO : save crop sub photo not yet done ! save time : 0.7281608581542969 nb_obj : 20 nb_hashtags : 4 time to prepare the origin masks : 8.267639875411987 time for calcul the mask position with numpy : 0.2998957633972168 nb_pixel_total : 7490857 time to create 1 rle with new method : 0.6224942207336426 time for calcul the mask position with numpy : 0.02509164810180664 nb_pixel_total : 58113 time to create 1 rle with old method : 0.06521916389465332 time for calcul the mask position with numpy : 0.04209160804748535 nb_pixel_total : 82902 time to create 1 rle with old method : 0.09039092063903809 time for calcul the mask position with numpy : 0.0447993278503418 nb_pixel_total : 9771 time to create 1 rle with old method : 0.01095438003540039 time for calcul the mask position with numpy : 0.042908430099487305 nb_pixel_total : 3607 time to create 1 rle with old method : 0.006226062774658203 time for calcul the mask position with numpy : 0.0440516471862793 nb_pixel_total : 122786 time to create 1 rle with old method : 0.15058541297912598 time for calcul the mask position with numpy : 0.04119133949279785 nb_pixel_total : 21675 time to create 1 rle with old method : 0.023880720138549805 time for calcul the mask position with numpy : 0.02898550033569336 nb_pixel_total : 16480 time to create 1 rle with old method : 0.01801586151123047 time for calcul the mask position with numpy : 0.027426481246948242 nb_pixel_total : 10912 time to create 1 rle with old method : 0.011776208877563477 time for calcul the mask position with numpy : 0.025376319885253906 nb_pixel_total : 24531 time to create 1 rle with old method : 0.026452064514160156 time for calcul the mask position with numpy : 0.02645134925842285 nb_pixel_total : 16053 time to create 1 rle with old method : 0.017589330673217773 time for calcul the mask position with numpy : 0.028332233428955078 nb_pixel_total : 11653 time to create 1 rle with old method : 0.014533758163452148 time for calcul the mask position with numpy : 0.02918553352355957 nb_pixel_total : 60559 time to create 1 rle with old method : 0.08226823806762695 time for calcul the mask position with numpy : 0.026866912841796875 nb_pixel_total : 95553 time to create 1 rle with old method : 0.10471534729003906 time for calcul the mask position with numpy : 0.029442548751831055 nb_pixel_total : 38077 time to create 1 rle with old method : 0.04658341407775879 time for calcul the mask position with numpy : 0.03078603744506836 nb_pixel_total : 32181 time to create 1 rle with old method : 0.0356907844543457 time for calcul the mask position with numpy : 0.02619338035583496 nb_pixel_total : 19563 time to create 1 rle with old method : 0.021776676177978516 time for calcul the mask position with numpy : 0.030116796493530273 nb_pixel_total : 20207 time to create 1 rle with old method : 0.022132158279418945 time for calcul the mask position with numpy : 0.02664470672607422 nb_pixel_total : 8911 time to create 1 rle with old method : 0.01019740104675293 time for calcul the mask position with numpy : 0.027938365936279297 nb_pixel_total : 83987 time to create 1 rle with old method : 0.11098766326904297 time for calcul the mask position with numpy : 0.02765059471130371 nb_pixel_total : 66022 time to create 1 rle with old method : 0.07237768173217773 create new chi : 2.5491321086883545 time to delete rle : 0.002377748489379883 batch 1 Loaded 41 chid ids of type : 3594 ++++++++++++++++++++++++++++++++Number RLEs to save : 12765 TO DO : save crop sub photo not yet done ! save time : 0.7262008190155029 nb_obj : 19 nb_hashtags : 2 time to prepare the origin masks : 7.293461799621582 time for calcul the mask position with numpy : 1.2734413146972656 nb_pixel_total : 7400324 time to create 1 rle with new method : 0.6106851100921631 time for calcul the mask position with numpy : 0.023624420166015625 nb_pixel_total : 15821 time to create 1 rle with old method : 0.018222332000732422 time for calcul the mask position with numpy : 0.027091503143310547 nb_pixel_total : 16679 time to create 1 rle with old method : 0.017992019653320312 time for calcul the mask position with numpy : 0.025487661361694336 nb_pixel_total : 34627 time to create 1 rle with old method : 0.03881025314331055 time for calcul the mask position with numpy : 0.027059316635131836 nb_pixel_total : 12213 time to create 1 rle with old method : 0.013577461242675781 time for calcul the mask position with numpy : 0.02604508399963379 nb_pixel_total : 5970 time to create 1 rle with old method : 0.007257938385009766 time for calcul the mask position with numpy : 0.026798486709594727 nb_pixel_total : 19681 time to create 1 rle with old method : 0.02208113670349121 time for calcul the mask position with numpy : 0.029864788055419922 nb_pixel_total : 144405 time to create 1 rle with old method : 0.19344663619995117 time for calcul the mask position with numpy : 0.029639482498168945 nb_pixel_total : 20066 time to create 1 rle with old method : 0.02419757843017578 time for calcul the mask position with numpy : 0.03183770179748535 nb_pixel_total : 38648 time to create 1 rle with old method : 0.05183100700378418 time for calcul the mask position with numpy : 0.03395819664001465 nb_pixel_total : 14039 time to create 1 rle with old method : 0.015951871871948242 time for calcul the mask position with numpy : 0.026865482330322266 nb_pixel_total : 3511 time to create 1 rle with old method : 0.004009246826171875 time for calcul the mask position with numpy : 0.03624582290649414 nb_pixel_total : 261066 time to create 1 rle with new method : 0.662581205368042 time for calcul the mask position with numpy : 0.025162935256958008 nb_pixel_total : 32687 time to create 1 rle with old method : 0.035455942153930664 time for calcul the mask position with numpy : 0.02552938461303711 nb_pixel_total : 58682 time to create 1 rle with old method : 0.06392240524291992 time for calcul the mask position with numpy : 0.028041601181030273 nb_pixel_total : 70176 time to create 1 rle with old method : 0.0750892162322998 time for calcul the mask position with numpy : 0.0265505313873291 nb_pixel_total : 34359 time to create 1 rle with old method : 0.04071927070617676 time for calcul the mask position with numpy : 0.027159690856933594 nb_pixel_total : 22870 time to create 1 rle with old method : 0.03709769248962402 time for calcul the mask position with numpy : 0.028267383575439453 nb_pixel_total : 42155 time to create 1 rle with old method : 0.04374957084655762 time for calcul the mask position with numpy : 0.024543046951293945 nb_pixel_total : 46421 time to create 1 rle with old method : 0.04992055892944336 create new chi : 3.9089460372924805 time to delete rle : 0.0019235610961914062 batch 1 Loaded 39 chid ids of type : 3594 ++++++++++++++++++++++++++++++Number RLEs to save : 13901 TO DO : save crop sub photo not yet done ! save time : 0.8529186248779297 map_output_result : {1336528979: (0.0, 'Should be the crop_list due to order', 0.0), 1336528884: (0.0, 'Should be the crop_list due to order', 0.0), 1336528786: (0.0, 'Should be the crop_list due to order', 0.0), 1336528724: (0.0, 'Should be the crop_list due to order', 0.0), 1336528189: (0.0, 'Should be the crop_list due to order', 0.0), 1336528104: (0.0, 'Should be the crop_list due to order', 0.0), 1336528004: (0.0, 'Should be the crop_list due to order', 0.0), 1336527904: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1336528979, 1336528884, 1336528786, 1336528724, 1336528189, 1336528104, 1336528004, 1336527904] Looping around the photos to save general results len do output : 8 /1336528979.Didn't retrieve data . /1336528884.Didn't retrieve data . /1336528786.Didn't retrieve data . /1336528724.Didn't retrieve data . /1336528189.Didn't retrieve data . /1336528104.Didn't retrieve data . /1336528004.Didn't retrieve data . /1336527904.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528979', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528884', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528786', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528724', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528189', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528104', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528004', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336527904', None, None, None, None, None, '2571901') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 24 time used for this insertion : 0.014955997467041016 save_final save missing photos in datou_result : time spend for datou_step_exec : 84.50866460800171 time spend to save output : 0.01570892333984375 total time spend for step 3 : 84.52437353134155 step4:ventilate_hashtags_in_portfolio Mon Feb 10 16:44:57 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 20410114 get user id for portfolio 20410114 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20410114 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('carton','flou','environnement','mal_croppe','pet_clair','pet_fonce','pehd','metal','background','autre','papier')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20410114 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('carton','flou','environnement','mal_croppe','pet_clair','pet_fonce','pehd','metal','background','autre','papier')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20410114 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('carton','flou','environnement','mal_croppe','pet_clair','pet_fonce','pehd','metal','background','autre','papier')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/20410541,20410542,20410543,20410544,20410545,20410546,20410547,20410548,20410549,20410550,20410551?tags=carton,flou,environnement,mal_croppe,pet_clair,pet_fonce,pehd,metal,background,autre,papier Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1336528979, 1336528884, 1336528786, 1336528724, 1336528189, 1336528104, 1336528004, 1336527904] Looping around the photos to save general results len do output : 1 /20410114. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528979', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528884', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528786', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528724', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528189', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528104', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528004', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336527904', None, None, None, None, None, '2571901') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.013207197189331055 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.5913701057434082 time spend to save output : 0.013431549072265625 total time spend for step 4 : 1.6048016548156738 step5:final Mon Feb 10 16:44:59 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1336528979: ('0.10845917124807102',), 1336528884: ('0.10845917124807102',), 1336528786: ('0.10845917124807102',), 1336528724: ('0.10845917124807102',), 1336528189: ('0.10845917124807102',), 1336528104: ('0.10845917124807102',), 1336528004: ('0.10845917124807102',), 1336527904: ('0.10845917124807102',)} new output for save of step final : {1336528979: ('0.10845917124807102',), 1336528884: ('0.10845917124807102',), 1336528786: ('0.10845917124807102',), 1336528724: ('0.10845917124807102',), 1336528189: ('0.10845917124807102',), 1336528104: ('0.10845917124807102',), 1336528004: ('0.10845917124807102',), 1336527904: ('0.10845917124807102',)} [1336528979, 1336528884, 1336528786, 1336528724, 1336528189, 1336528104, 1336528004, 1336527904] Looping around the photos to save general results len do output : 8 /1336528979.Didn't retrieve data . /1336528884.Didn't retrieve data . /1336528786.Didn't retrieve data . /1336528724.Didn't retrieve data . /1336528189.Didn't retrieve data . /1336528104.Didn't retrieve data . /1336528004.Didn't retrieve data . /1336527904.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528979', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528884', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528786', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528724', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528189', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528104', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528004', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336527904', None, None, None, None, None, '2571901') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 24 time used for this insertion : 0.01326441764831543 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.12189006805419922 time spend to save output : 0.013852119445800781 total time spend for step 5 : 0.1357421875 step6:blur_detection Mon Feb 10 16:44:59 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9.jpg resize: (2160, 3840) 1336528979 -5.358109105929672 treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84.jpg resize: (2160, 3840) 1336528884 -5.866458529972003 treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972.jpg resize: (2160, 3840) 1336528786 -5.616767432020546 treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2.jpg resize: (2160, 3840) 1336528724 -5.283541728890685 treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d.jpg resize: (2160, 3840) 1336528189 -5.443571455124416 treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2.jpg resize: (2160, 3840) 1336528104 -5.451844578733759 treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440.jpg resize: (2160, 3840) 1336528004 -6.067650190123516 treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de.jpg resize: (2160, 3840) 1336527904 -5.907066023810714 treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213594_0.png resize: (251, 253) 1336536099 -3.811905477189072 treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213586_0.png resize: (522, 301) 1336536102 -3.6464141117326285 treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213583_0.png resize: (226, 271) 1336536103 -2.834916717484657 treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213589_0.png resize: (424, 247) 1336536104 -3.636279630309849 treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213578_0.png resize: (485, 416) 1336536105 -2.7265798498200944 treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213585_0.png resize: (171, 240) 1336536106 -2.640508904534311 treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213588_0.png resize: (211, 333) 1336536107 -4.086844590572499 treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213582_0.png resize: (295, 409) 1336536108 -2.7635243731334085 treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213593_0.png resize: (235, 241) 1336536109 -2.4428585945131873 treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213577_0.png resize: (210, 206) 1336536110 -1.7339669291964086 treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213592_0.png resize: (205, 242) 1336536111 -4.132194706869208 treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213595_0.png resize: (165, 163) 1336536112 -2.4705476299549884 treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213591_0.png resize: (72, 199) 1336536113 -3.6500538668745777 treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213579_0.png resize: (258, 335) 1336536114 -3.2760706755747546 treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213580_0.png resize: (182, 265) 1336536115 -3.1870106436662975 treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213605_0.png resize: (256, 104) 1336536116 1.148123757753792 treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213608_0.png resize: (294, 369) 1336536117 -2.8171874228075424 treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213606_0.png resize: (216, 178) 1336536118 -1.8183252872663476 treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213604_0.png resize: (291, 183) 1336536119 -2.0369318867185386 treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213603_0.png resize: (265, 247) 1336536120 -3.652202006215548 treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213600_0.png resize: (276, 157) 1336536121 -3.3777434143936387 treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213607_0.png resize: (193, 176) 1336536122 -3.454263264197937 treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213597_0.png resize: (170, 210) 1336536123 -3.2038962861066427 treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213611_0.png resize: (105, 100) 1336536124 -4.575039597966971 treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213609_0.png resize: (618, 277) 1336536125 -3.087287906611433 treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213602_0.png resize: (282, 164) 1336536126 -3.663554462737826 treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213598_0.png resize: (473, 254) 1336536127 -4.212415806871047 treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213596_0.png resize: (128, 139) 1336536128 -0.4576748061789381 treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213601_0.png resize: (202, 247) 1336536129 -2.0789287094868323 treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213623_0.png resize: (859, 965) 1336536130 -4.137325837880134 treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213619_0.png resize: (301, 288) 1336536131 -2.008858651214861 treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213620_0.png resize: (313, 290) 1336536132 -2.7997937612359634 treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213625_0.png resize: (161, 239) 1336536133 -2.399794542985859 treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213613_0.png resize: (223, 265) 1336536134 -3.140828532932747 treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213621_0.png resize: (418, 318) 1336536135 -1.3071092905586665 treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213624_0.png resize: (638, 316) 1336536136 -4.291908883974006 treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213629_0.png resize: (153, 84) 1336536137 -3.0319168128612675 treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213617_0.png resize: (136, 166) 1336536138 -3.4350059478814225 treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213622_0.png resize: (151, 103) 1336536139 -3.670380131151406 treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213626_0.png resize: (346, 313) 1336536140 -1.6212045692713328 treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213616_0.png resize: (277, 203) 1336536141 -2.066171697940601 treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213628_0.png resize: (296, 155) 1336536142 -0.6410863177125388 treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213630_0.png resize: (219, 137) 1336536143 -3.7099671384988815 treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213638_0.png resize: (303, 248) 1336536144 -3.431992647845196 treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213631_0.png resize: (233, 222) 1336536145 -2.6933650250007175 treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213640_0.png resize: (443, 556) 1336536146 -4.3640727773418195 treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213643_0.png resize: (291, 199) 1336536147 -2.253845171120623 treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213644_0.png resize: (185, 153) 1336536148 -3.1161648738288092 treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213634_0.png resize: (125, 164) 1336536149 -2.4756187224090964 treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213639_0.png resize: (199, 216) 1336536150 -2.3753515947658745 treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213647_0.png resize: (342, 340) 1336536151 -2.724750773683438 treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213641_0.png resize: (291, 167) 1336536152 -2.359556525421629 treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213637_0.png resize: (198, 183) 1336536153 -2.551126429732559 treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213648_0.png resize: (139, 189) 1336536154 -0.9561442929578309 treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213649_0.png resize: (253, 301) 1336536155 -3.43114145900819 treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213633_0.png resize: (141, 318) 1336536156 -3.750388468164724 treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213635_0.png resize: (197, 112) 1336536157 -2.497125566173613 treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213651_0.png resize: (397, 277) 1336536158 -3.0475613322393036 treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213642_0.png resize: (255, 141) 1336536159 -3.096975699653762 treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213661_0.png resize: (793, 892) 1336536160 -4.87309825469702 treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213662_0.png resize: (116, 102) 1336536161 -0.9371061769295747 treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213676_0.png resize: (145, 140) 1336536162 -3.71204122799158 treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213660_0.png resize: (111, 68) 1336536163 -2.3414973711391625 treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213663_0.png resize: (310, 277) 1336536164 -2.291919392159482 treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213656_0.png resize: (260, 231) 1336536165 -3.031605265695101 treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213664_0.png resize: (271, 226) 1336536166 -2.9718077777382566 treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213670_0.png resize: (262, 289) 1336536167 -3.319394815005453 treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213668_0.png resize: (366, 281) 1336536168 -2.8760385273439097 treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213665_0.png resize: (173, 197) 1336536170 -3.1918336242222085 treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213653_0.png resize: (220, 311) 1336536171 -2.3786056461982117 treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213657_0.png resize: (198, 161) 1336536172 -3.4821515477247678 treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213669_0.png resize: (159, 110) 1336536173 -1.4726839296892373 treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213655_0.png resize: (196, 211) 1336536174 -3.3652760972467934 treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213673_0.png resize: (173, 161) 1336536175 -2.3020543486522502 treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213674_0.png resize: (209, 198) 1336536176 -3.4619980750589177 treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213666_0.png resize: (228, 174) 1336536177 -3.401018706870771 treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213658_0.png resize: (264, 173) 1336536178 -2.8414727451567794 treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213672_0.png resize: (182, 210) 1336536179 -1.153758236727043 treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213677_0.png resize: (190, 113) 1336536180 -1.9397647805216758 treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213671_0.png resize: (181, 225) 1336536181 -1.8181681630869253 treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213690_0.png resize: (305, 174) 1336536182 -2.3989344990734445 treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213683_0.png resize: (148, 132) 1336536183 -2.4245918027257343 treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213689_0.png resize: (384, 554) 1336536184 -4.245085792100747 treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213679_0.png resize: (178, 218) 1336536185 -2.5543970235925495 treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213692_0.png resize: (167, 153) 1336536186 -2.1687239236638662 treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213687_0.png resize: (163, 147) 1336536187 -3.645153973477387 treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213694_0.png resize: (135, 206) 1336536188 -4.163802504826645 treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213684_0.png resize: (451, 292) 1336536189 -2.8724713290659043 treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213697_0.png resize: (117, 57) 1336536190 -3.7813480670687754 treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213686_0.png resize: (80, 87) 1336536191 -1.9852498477580003 treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213682_0.png resize: (415, 485) 1336536192 -3.460261884706478 treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213680_0.png resize: (348, 418) 1336536193 -3.466907673596187 treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213696_0.png resize: (121, 205) 1336536194 -3.0148351213596345 treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213693_0.png resize: (240, 143) 1336536195 -2.777511478481336 treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213681_0.png resize: (289, 215) 1336536196 -3.3966287467982013 treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213685_0.png resize: (236, 251) 1336536197 -3.0912524500844705 treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213691_0.png resize: (263, 126) 1336536198 -2.078132531979795 treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213688_0.png resize: (123, 151) 1336536199 -1.7381058910748575 treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213714_0.png resize: (291, 243) 1336536200 -2.7141678809856096 treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213713_0.png resize: (491, 373) 1336536201 -3.9213208676431206 treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213700_0.png resize: (170, 117) 1336536202 -3.142806649952932 treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213708_0.png resize: (140, 149) 1336536203 -3.1263235622940213 treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213715_0.png resize: (179, 81) 1336536204 -3.6682283973499223 treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213710_0.png resize: (136, 115) 1336536205 -3.4764380791038354 treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213717_0.png resize: (200, 341) 1336536206 -1.1369812865309759 treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213709_0.png resize: (275, 121) 1336536207 -2.5638728733463942 treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213703_0.png resize: (256, 154) 1336536208 -1.8355381428038664 treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213701_0.png resize: (199, 159) 1336536209 -0.6873861287709604 treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213712_0.png resize: (245, 123) 1336536210 -3.6144690679531952 treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213707_0.png resize: (181, 134) 1336536212 -3.0692035477480624 treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213733_0.png resize: (226, 229) 1336536213 -2.0689810592380193 treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213725_0.png resize: (551, 797) 1336536214 -3.8881242752492526 treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213718_0.png resize: (189, 330) 1336536215 -2.6204868849093805 treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213719_0.png resize: (233, 243) 1336536216 -1.6445622601764955 treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213726_0.png resize: (188, 163) 1336536217 -3.72458181520105 treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213732_0.png resize: (159, 199) 1336536218 -3.5829410600924074 treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213724_0.png resize: (219, 192) 1336536219 -2.346951093760738 treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213720_0.png resize: (230, 167) 1336536220 -3.159315707971932 treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213721_0.png resize: (244, 196) 1336536221 -2.233572520540737 treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213723_0.png resize: (409, 321) 1336536222 -3.0628634382345434 treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213736_0.png resize: (163, 186) 1336536223 -2.7176349003428313 treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213730_0.png resize: (530, 520) 1336536224 -2.1413892945344695 treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213731_0.png resize: (515, 92) 1336536225 -3.3473292978196603 treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213735_0.png resize: (128, 172) 1336536226 -2.1445445468784814 treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213587_0.png resize: (478, 194) 1336536235 -4.29576048447024 treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213581_0.png resize: (261, 205) 1336536236 -3.368960245677795 treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213615_0.png resize: (197, 251) 1336536237 -3.0810091959312382 treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213614_0.png resize: (164, 174) 1336536238 -2.5052956608870405 treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213627_0.png resize: (149, 111) 1336536239 -3.3798620133749684 treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213632_0.png resize: (174, 149) 1336536240 -2.542470838500601 treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213650_0.png resize: (280, 254) 1336536241 -3.8229174049740005 treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213646_0.png resize: (165, 204) 1336536242 -2.9280909708667497 treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213636_0.png resize: (211, 175) 1336536243 -2.926642048076479 treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213645_0.png resize: (199, 130) 1336536244 -3.7185709824109274 treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213654_0.png resize: (218, 222) 1336536245 -2.749307761113725 treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213667_0.png resize: (155, 137) 1336536246 -2.5632308864935274 treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213652_0.png resize: (222, 290) 1336536248 -3.2512636869511233 treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213659_0.png resize: (177, 211) 1336536249 -2.8382948356326243 treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213678_0.png resize: (409, 220) 1336536250 -2.482061573132784 treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213695_0.png resize: (200, 130) 1336536251 -3.8213682295726907 treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213704_0.png resize: (286, 232) 1336536252 -3.0692032895056065 treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213706_0.png resize: (225, 344) 1336536254 -2.8037402277177463 treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213702_0.png resize: (227, 149) 1336536255 -3.586116029985538 treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213711_0.png resize: (125, 218) 1336536256 -3.9945259750061726 treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213722_0.png resize: (271, 366) 1336536257 -3.399267020317458 treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213728_0.png resize: (179, 281) 1336536258 -3.4922767604742866 treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213729_0.png resize: (250, 123) 1336536259 -4.043315448760066 treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213734_0.png resize: (313, 192) 1336536260 -3.3029001714138246 treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213727_0.png resize: (205, 100) 1336536261 -2.9195638176032146 treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213590_0.png resize: (212, 125) 1336536262 -3.6694627350377886 treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213675_0.png resize: (272, 211) 1336536263 -4.343890044090375 treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213610_0.png resize: (497, 350) 1336536266 -3.1895535165320106 treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213599_0.png resize: (444, 294) 1336536267 -3.2931168778589086 treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213618_0.png resize: (235, 314) 1336536268 -2.169560305049374 treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213698_0.png resize: (201, 418) 1336536269 -2.8747074798135834 treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213705_0.png resize: (364, 356) 1336536271 -2.7723494231596275 treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213699_0.png resize: (335, 387) 1336536272 -3.9587397312342882 treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213584_0.png resize: (115, 139) 1336536275 -3.2977302845642114 treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213612_0.png resize: (285, 370) 1336536276 -4.39968307549002 treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213716_0.png resize: (517, 279) 1336536277 -4.185255007388245 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 168 time used for this insertion : 0.019328832626342773 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 168 time used for this insertion : 0.11643195152282715 save missing photos in datou_result : time spend for datou_step_exec : 29.649879455566406 time spend to save output : 0.1411113739013672 total time spend for step 6 : 29.790990829467773 step7:brightness Mon Feb 10 16:45:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9.jpg treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84.jpg treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972.jpg treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2.jpg treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d.jpg treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2.jpg treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440.jpg treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de.jpg treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213594_0.png treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213586_0.png treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213583_0.png treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213589_0.png treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213578_0.png treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213585_0.png treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213588_0.png treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213582_0.png treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213593_0.png treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213577_0.png treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213592_0.png treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213595_0.png treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213591_0.png treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213579_0.png treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213580_0.png treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213605_0.png treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213608_0.png treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213606_0.png treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213604_0.png treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213603_0.png treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213600_0.png treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213607_0.png treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213597_0.png treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213611_0.png treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213609_0.png treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213602_0.png treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213598_0.png treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213596_0.png treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213601_0.png treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213623_0.png treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213619_0.png treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213620_0.png treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213625_0.png treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213613_0.png treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213621_0.png treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213624_0.png treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213629_0.png treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213617_0.png treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213622_0.png treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213626_0.png treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213616_0.png treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213628_0.png treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213630_0.png treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213638_0.png treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213631_0.png treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213640_0.png treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213643_0.png treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213644_0.png treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213634_0.png treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213639_0.png treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213647_0.png treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213641_0.png treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213637_0.png treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213648_0.png treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213649_0.png treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213633_0.png treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213635_0.png treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213651_0.png treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213642_0.png treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213661_0.png treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213662_0.png treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213676_0.png treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213660_0.png treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213663_0.png treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213656_0.png treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213664_0.png treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213670_0.png treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213668_0.png treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213665_0.png treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213653_0.png treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213657_0.png treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213669_0.png treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213655_0.png treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213673_0.png treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213674_0.png treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213666_0.png treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213658_0.png treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213672_0.png treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213677_0.png treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213671_0.png treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213690_0.png treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213683_0.png treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213689_0.png treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213679_0.png treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213692_0.png treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213687_0.png treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213694_0.png treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213684_0.png treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213697_0.png treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213686_0.png treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213682_0.png treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213680_0.png treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213696_0.png treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213693_0.png treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213681_0.png treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213685_0.png treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213691_0.png treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213688_0.png treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213714_0.png treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213713_0.png treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213700_0.png treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213708_0.png treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213715_0.png treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213710_0.png treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213717_0.png treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213709_0.png treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213703_0.png treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213701_0.png treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213712_0.png treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213707_0.png treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213733_0.png treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213725_0.png treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213718_0.png treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213719_0.png treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213726_0.png treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213732_0.png treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213724_0.png treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213720_0.png treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213721_0.png treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213723_0.png treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213736_0.png treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213730_0.png treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213731_0.png treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213735_0.png treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213587_0.png treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213581_0.png treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213615_0.png treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213614_0.png treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213627_0.png treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213632_0.png treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213650_0.png treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213646_0.png treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213636_0.png treat image : temp/1739202029_3153736_1336528724_72f44d8e18e63dee56cc035fbca80fb2_rle_crop_3668213645_0.png treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213654_0.png treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213667_0.png treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213652_0.png treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213659_0.png treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213678_0.png treat image : temp/1739202029_3153736_1336528104_9817d36f6d3f1a3a07303e751b513de2_rle_crop_3668213695_0.png treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213704_0.png treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213706_0.png treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213702_0.png treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213711_0.png treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213722_0.png treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213728_0.png treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213729_0.png treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213734_0.png treat image : temp/1739202029_3153736_1336527904_c5f0aded9da83a86979ba2cfad2302de_rle_crop_3668213727_0.png treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213590_0.png treat image : temp/1739202029_3153736_1336528189_d98d08cd1ea36b2df55559e29944659d_rle_crop_3668213675_0.png treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213610_0.png treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213599_0.png treat image : temp/1739202029_3153736_1336528786_5aae9e4ff7106b98e7009e085e453972_rle_crop_3668213618_0.png treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213698_0.png treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213705_0.png treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213699_0.png treat image : temp/1739202029_3153736_1336528979_74e46dd5df358ab290d23950663d7ba9_rle_crop_3668213584_0.png treat image : temp/1739202029_3153736_1336528884_5f42f834d8c0b9e5d4bec0fcd9956d84_rle_crop_3668213612_0.png treat image : temp/1739202029_3153736_1336528004_b0cd7a7134f84db83d0e5ebc8edad440_rle_crop_3668213716_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 168 time used for this insertion : 0.01609206199645996 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 168 time used for this insertion : 0.0301361083984375 save missing photos in datou_result : time spend for datou_step_exec : 9.311725616455078 time spend to save output : 0.05151486396789551 total time spend for step 7 : 9.363240480422974 step8:velours_tree Mon Feb 10 16:45:38 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.9253928661346436 time spend to save output : 3.838539123535156e-05 total time spend for step 8 : 0.9254312515258789 step9:send_mail_cod Mon Feb 10 16:45:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P20410114_10-02-2025_16_45_39.pdf 20410541 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette204105411739202339 20410542 imagette204105421739202340 20410544 imagette204105441739202340 20410545 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette204105451739202340 20410546 imagette204105461739202341 20410547 change filename to text .imagette204105471739202341 20410548 change filename to text .change filename to text .imagette204105481739202341 20410549 imagette204105491739202341 20410550 change filename to text .change filename to text .imagette204105501739202341 20410551 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette204105511739202341 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20410114 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20410541,20410542,20410543,20410544,20410545,20410546,20410547,20410548,20410549,20410550,20410551?tags=carton,flou,environnement,mal_croppe,pet_clair,pet_fonce,pehd,metal,background,autre,papier args[1336528979] : ((1336528979, -5.358109105929672, 492609224), (1336528979, -0.16263801229986283, 496442774), '0.10845917124807102') apple ((1336528979, -5.358109105929672, 492609224), (1336528979, -0.16263801229986283, 496442774), '0.10845917124807102') We are sending mail with results at report@fotonower.com args[1336528884] : ((1336528884, -5.866458529972003, 492609224), (1336528884, 0.7324856256540794, 2107752395), '0.10845917124807102') apple ((1336528884, -5.866458529972003, 492609224), (1336528884, 0.7324856256540794, 2107752395), '0.10845917124807102') We are sending mail with results at report@fotonower.com args[1336528786] : ((1336528786, -5.616767432020546, 492609224), (1336528786, 0.13291748065800763, 2107752395), '0.10845917124807102') apple ((1336528786, -5.616767432020546, 492609224), (1336528786, 0.13291748065800763, 2107752395), '0.10845917124807102') We are sending mail with results at report@fotonower.com args[1336528724] : ((1336528724, -5.283541728890685, 492609224), (1336528724, -0.15878922737642095, 496442774), '0.10845917124807102') apple ((1336528724, -5.283541728890685, 492609224), (1336528724, -0.15878922737642095, 496442774), '0.10845917124807102') We are sending mail with results at report@fotonower.com args[1336528189] : ((1336528189, -5.443571455124416, 492609224), (1336528189, -0.16965089167307218, 496442774), '0.10845917124807102') apple ((1336528189, -5.443571455124416, 492609224), (1336528189, -0.16965089167307218, 496442774), '0.10845917124807102') We are sending mail with results at report@fotonower.com args[1336528104] : ((1336528104, -5.451844578733759, 492609224), (1336528104, 0.050209505883951455, 2107752395), '0.10845917124807102') apple ((1336528104, -5.451844578733759, 492609224), (1336528104, 0.050209505883951455, 2107752395), '0.10845917124807102') We are sending mail with results at report@fotonower.com args[1336528004] : ((1336528004, -6.067650190123516, 492609224), (1336528004, 0.39315149828917234, 2107752395), '0.10845917124807102') apple ((1336528004, -6.067650190123516, 492609224), (1336528004, 0.39315149828917234, 2107752395), '0.10845917124807102') We are sending mail with results at report@fotonower.com args[1336527904] : ((1336527904, -5.907066023810714, 492609224), (1336527904, 0.5968633486121593, 2107752395), '0.10845917124807102') apple ((1336527904, -5.907066023810714, 492609224), (1336527904, 0.5968633486121593, 2107752395), '0.10845917124807102') We are sending mail with results at report@fotonower.com refus_total : 0.10845917124807102 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20410114 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20410114_10-02-2025_16_45_39.pdf results_Auto_P20410114_10-02-2025_16_45_39.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20410114_10-02-2025_16_45_39.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','20410114','results_Auto_P20410114_10-02-2025_16_45_39.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20410114_10-02-2025_16_45_39.pdf','pdf','','0.6','0.10845917124807102') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/20410114

https://www.fotonower.com/image?json=false&list_photos_id=1336528979
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1336528884
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1336528786
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1336528724
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1336528189
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1336528104
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1336528004
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1336527904
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 10.85%
Veuillez trouver les photos des contaminants.

exemples de contaminants: carton: https://www.fotonower.com/view/20410541?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/20410545?limit=200
exemples de contaminants: pehd: https://www.fotonower.com/view/20410547?limit=200
exemples de contaminants: metal: https://www.fotonower.com/view/20410548?limit=200
exemples de contaminants: autre: https://www.fotonower.com/view/20410550?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/20410551?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20410114_10-02-2025_16_45_39.pdf.

Lien vers velours :https://www.fotonower.com/velours/20410541,20410542,20410543,20410544,20410545,20410546,20410547,20410548,20410549,20410550,20410551?tags=carton,flou,environnement,mal_croppe,pet_clair,pet_fonce,pehd,metal,background,autre,papier.


L'équipe Fotonower 202 b'' Server: nginx Date: Mon, 10 Feb 2025 15:45:45 GMT Content-Length: 0 Connection: close X-Message-Id: A9187O4sQIuSWLSY3_Oq_g Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1336528979, 1336528884, 1336528786, 1336528724, 1336528189, 1336528104, 1336528004, 1336527904] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528979', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528884', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528786', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528724', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528189', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528104', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528004', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336527904', None, None, None, None, None, '2571901') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 8 time used for this insertion : 0.012583255767822266 save_final save missing photos in datou_result : time spend for datou_step_exec : 5.7864603996276855 time spend to save output : 0.012781620025634766 total time spend for step 9 : 5.79924201965332 step10:split_time_score Mon Feb 10 16:45:45 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('16', 8),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 10022025 20410114 Nombre de photos uploadées : 8 / 23040 (0%) 10022025 20410114 Nombre de photos taguées (types de déchets): 0 / 8 (0%) 10022025 20410114 Nombre de photos taguées (volume) : 0 / 8 (0%) elapsed_time : load_data_split_time_score 2.1457672119140625e-06 elapsed_time : order_list_meta_photo_and_scores 5.0067901611328125e-06 ???????? elapsed_time : fill_and_build_computed_from_old_data 0.000400543212890625 elapsed_time : insert_dashboard_record_day_entry 0.025908708572387695 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.09136817210005146 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20395415_10-02-2025_09_50_05.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20395415 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20395415 AND mptpi.`type`=3594 To do Qualite : 0.10350550485192844 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20398813_10-02-2025_11_04_37.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20398813 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20398813 AND mptpi.`type`=3726 To do Qualite : 0.01134759147128439 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20402618_10-02-2025_12_36_42.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20402618 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20402618 AND mptpi.`type`=3726 To do Qualite : 0.02496931143484574 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20403540_10-02-2025_13_49_26.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20403540 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20403540 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20410072 order by id desc limit 1 Qualite : 0.009208106012841779 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20410113_10-02-2025_16_36_41.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20410113 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20410113 AND mptpi.`type`=3726 To do Qualite : 0.10845917124807102 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20410114_10-02-2025_16_45_39.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20410114 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20410114 AND mptpi.`type`=3594 To do Qualite : 0.03331702054486747 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20410115_10-02-2025_16_41_17.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20410115 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20410115 AND mptpi.`type`=3726 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'10022025': {'nb_upload': 8, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1336528979, 1336528884, 1336528786, 1336528724, 1336528189, 1336528104, 1336528004, 1336527904] Looping around the photos to save general results len do output : 1 /20410114Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528979', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528884', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528786', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528724', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528189', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528104', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336528004', None, None, None, None, None, '2571901') ('3318', None, None, None, None, None, None, None, '2571901') ('3318', '20410114', '1336527904', None, None, None, None, None, '2571901') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.01284933090209961 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.9223339557647705 time spend to save output : 0.013096332550048828 total time spend for step 10 : 1.9354302883148193 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 8 set_done_treatment 151.17user 115.48system 5:21.95elapsed 82%CPU (0avgtext+0avgdata 5796716maxresident)k 614992inputs+119264outputs (9872major+14412227minor)pagefaults 0swaps