python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 174439 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['2593986'] with mtr_portfolio_ids : ['20615073'] and first list_photo_ids : [] new path : /proc/174439/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 5 ; length of list_pids : 5 ; length of list_args : 5 time to download the photos : 0.9237799644470215 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Mon Feb 17 17:40:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10998 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-17 17:40:32.473933: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-17 17:40:32.499037: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-17 17:40:32.500467: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f5254000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-17 17:40:32.500523: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-17 17:40:32.503163: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-17 17:40:32.810112: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x39117d80 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-17 17:40:32.810162: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-17 17:40:32.811088: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-17 17:40:32.811402: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-17 17:40:32.816929: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-17 17:40:32.818968: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-17 17:40:32.819292: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-17 17:40:32.821522: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-17 17:40:32.822516: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-17 17:40:32.826894: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-17 17:40:32.828453: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-17 17:40:32.828525: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-17 17:40:32.829273: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-17 17:40:32.829288: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-17 17:40:32.829298: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-17 17:40:32.830610: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10193 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-17 17:40:33.082020: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-17 17:40:33.082117: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-17 17:40:33.082139: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-17 17:40:33.082159: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-17 17:40:33.082178: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-17 17:40:33.082196: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-17 17:40:33.082215: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-17 17:40:33.082235: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-17 17:40:33.083903: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-17 17:40:33.085208: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-17 17:40:33.085243: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-17 17:40:33.085263: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-17 17:40:33.085281: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-17 17:40:33.085299: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-17 17:40:33.085317: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-17 17:40:33.085336: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-17 17:40:33.085354: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-17 17:40:33.087174: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-17 17:40:33.087207: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-17 17:40:33.087217: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-17 17:40:33.087227: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-17 17:40:33.088666: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10193 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-17 17:40:43.124743: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-17 17:40:43.294133: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 5 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 141.85000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 31 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 140.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 47 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 141.35000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 41 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 2.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 139.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 33 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 3.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 23 Detection mask done ! Trying to reset tf kernel 174842 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5706 tf kernel not reseted sub process len(results) : 5 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 5 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10998 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0011110305786132812 nb_pixel_total : 37513 time to create 1 rle with old method : 0.040266990661621094 length of segment : 327 time for calcul the mask position with numpy : 0.0006725788116455078 nb_pixel_total : 39088 time to create 1 rle with old method : 0.042354583740234375 length of segment : 229 time for calcul the mask position with numpy : 0.0026688575744628906 nb_pixel_total : 121094 time to create 1 rle with old method : 0.12808918952941895 length of segment : 437 time for calcul the mask position with numpy : 0.0035982131958007812 nb_pixel_total : 140088 time to create 1 rle with old method : 0.14814209938049316 length of segment : 563 time for calcul the mask position with numpy : 0.0005011558532714844 nb_pixel_total : 27050 time to create 1 rle with old method : 0.028690338134765625 length of segment : 291 time for calcul the mask position with numpy : 0.0004565715789794922 nb_pixel_total : 29156 time to create 1 rle with old method : 0.03112626075744629 length of segment : 243 time for calcul the mask position with numpy : 0.0008685588836669922 nb_pixel_total : 51746 time to create 1 rle with old method : 0.05513191223144531 length of segment : 383 time for calcul the mask position with numpy : 0.00034928321838378906 nb_pixel_total : 19172 time to create 1 rle with old method : 0.021760225296020508 length of segment : 196 time for calcul the mask position with numpy : 0.0004918575286865234 nb_pixel_total : 22228 time to create 1 rle with old method : 0.028868675231933594 length of segment : 176 time for calcul the mask position with numpy : 0.00041675567626953125 nb_pixel_total : 18760 time to create 1 rle with old method : 0.020960330963134766 length of segment : 203 time for calcul the mask position with numpy : 0.0009622573852539062 nb_pixel_total : 43727 time to create 1 rle with old method : 0.05707216262817383 length of segment : 346 time for calcul the mask position with numpy : 0.00044798851013183594 nb_pixel_total : 28115 time to create 1 rle with old method : 0.03018975257873535 length of segment : 187 time for calcul the mask position with numpy : 0.0008931159973144531 nb_pixel_total : 53655 time to create 1 rle with old method : 0.06017923355102539 length of segment : 490 time for calcul the mask position with numpy : 0.0004482269287109375 nb_pixel_total : 24440 time to create 1 rle with old method : 0.02770853042602539 length of segment : 190 time for calcul the mask position with numpy : 0.0007026195526123047 nb_pixel_total : 33866 time to create 1 rle with old method : 0.03765702247619629 length of segment : 207 time for calcul the mask position with numpy : 0.0006821155548095703 nb_pixel_total : 33548 time to create 1 rle with old method : 0.037966251373291016 length of segment : 317 time for calcul the mask position with numpy : 0.00035643577575683594 nb_pixel_total : 18103 time to create 1 rle with old method : 0.019930601119995117 length of segment : 149 time for calcul the mask position with numpy : 0.0006651878356933594 nb_pixel_total : 32088 time to create 1 rle with old method : 0.036109209060668945 length of segment : 264 time for calcul the mask position with numpy : 0.0010485649108886719 nb_pixel_total : 47017 time to create 1 rle with old method : 0.051721811294555664 length of segment : 359 time for calcul the mask position with numpy : 0.0024678707122802734 nb_pixel_total : 124097 time to create 1 rle with old method : 0.1318378448486328 length of segment : 575 time for calcul the mask position with numpy : 0.00070953369140625 nb_pixel_total : 33145 time to create 1 rle with old method : 0.034177541732788086 length of segment : 317 time for calcul the mask position with numpy : 0.0010035037994384766 nb_pixel_total : 51547 time to create 1 rle with old method : 0.05280733108520508 length of segment : 384 time for calcul the mask position with numpy : 0.00034332275390625 nb_pixel_total : 18547 time to create 1 rle with old method : 0.01937699317932129 length of segment : 195 time for calcul the mask position with numpy : 0.0010814666748046875 nb_pixel_total : 39091 time to create 1 rle with old method : 0.047267913818359375 length of segment : 239 time for calcul the mask position with numpy : 0.0009760856628417969 nb_pixel_total : 49560 time to create 1 rle with old method : 0.060433149337768555 length of segment : 199 time for calcul the mask position with numpy : 0.0006814002990722656 nb_pixel_total : 33567 time to create 1 rle with old method : 0.0372467041015625 length of segment : 187 time for calcul the mask position with numpy : 0.00096893310546875 nb_pixel_total : 57707 time to create 1 rle with old method : 0.06501412391662598 length of segment : 377 time for calcul the mask position with numpy : 0.0003662109375 nb_pixel_total : 22950 time to create 1 rle with old method : 0.026237010955810547 length of segment : 176 time for calcul the mask position with numpy : 0.0018558502197265625 nb_pixel_total : 79428 time to create 1 rle with old method : 0.09130311012268066 length of segment : 458 time for calcul the mask position with numpy : 0.0012180805206298828 nb_pixel_total : 53112 time to create 1 rle with old method : 0.05804753303527832 length of segment : 500 time for calcul the mask position with numpy : 0.0024657249450683594 nb_pixel_total : 117722 time to create 1 rle with old method : 0.1315314769744873 length of segment : 277 time for calcul the mask position with numpy : 0.0005409717559814453 nb_pixel_total : 22509 time to create 1 rle with old method : 0.025908470153808594 length of segment : 226 time for calcul the mask position with numpy : 0.0008587837219238281 nb_pixel_total : 53030 time to create 1 rle with old method : 0.07200360298156738 length of segment : 388 time for calcul the mask position with numpy : 0.0006663799285888672 nb_pixel_total : 31348 time to create 1 rle with old method : 0.03556203842163086 length of segment : 212 time for calcul the mask position with numpy : 0.0003466606140136719 nb_pixel_total : 16400 time to create 1 rle with old method : 0.018321514129638672 length of segment : 111 time for calcul the mask position with numpy : 0.0002665519714355469 nb_pixel_total : 11087 time to create 1 rle with old method : 0.012581110000610352 length of segment : 119 time for calcul the mask position with numpy : 0.0010526180267333984 nb_pixel_total : 45515 time to create 1 rle with old method : 0.049806833267211914 length of segment : 321 time for calcul the mask position with numpy : 0.0016508102416992188 nb_pixel_total : 45646 time to create 1 rle with old method : 0.05018019676208496 length of segment : 640 time for calcul the mask position with numpy : 0.0003364086151123047 nb_pixel_total : 10243 time to create 1 rle with old method : 0.011728763580322266 length of segment : 160 time for calcul the mask position with numpy : 0.0005702972412109375 nb_pixel_total : 23850 time to create 1 rle with old method : 0.0253756046295166 length of segment : 214 time for calcul the mask position with numpy : 0.00043582916259765625 nb_pixel_total : 15400 time to create 1 rle with old method : 0.0164487361907959 length of segment : 181 time for calcul the mask position with numpy : 0.0009191036224365234 nb_pixel_total : 40400 time to create 1 rle with old method : 0.04219555854797363 length of segment : 280 time for calcul the mask position with numpy : 0.000331878662109375 nb_pixel_total : 14114 time to create 1 rle with old method : 0.019897937774658203 length of segment : 207 time for calcul the mask position with numpy : 0.024529218673706055 nb_pixel_total : 601125 time to create 1 rle with new method : 0.3202950954437256 length of segment : 1869 time for calcul the mask position with numpy : 0.003159046173095703 nb_pixel_total : 63831 time to create 1 rle with old method : 0.06702685356140137 length of segment : 366 time for calcul the mask position with numpy : 0.004110097885131836 nb_pixel_total : 98837 time to create 1 rle with old method : 0.11265778541564941 length of segment : 374 time for calcul the mask position with numpy : 0.0014410018920898438 nb_pixel_total : 30543 time to create 1 rle with old method : 0.033631086349487305 length of segment : 245 time for calcul the mask position with numpy : 0.00144195556640625 nb_pixel_total : 35068 time to create 1 rle with old method : 0.038969993591308594 length of segment : 279 time for calcul the mask position with numpy : 0.0035986900329589844 nb_pixel_total : 86089 time to create 1 rle with old method : 0.09601593017578125 length of segment : 289 time for calcul the mask position with numpy : 0.0018825531005859375 nb_pixel_total : 41125 time to create 1 rle with old method : 0.04839515686035156 length of segment : 483 time for calcul the mask position with numpy : 0.0027649402618408203 nb_pixel_total : 56477 time to create 1 rle with old method : 0.06412506103515625 length of segment : 520 time for calcul the mask position with numpy : 0.002668619155883789 nb_pixel_total : 54600 time to create 1 rle with old method : 0.05994057655334473 length of segment : 385 time for calcul the mask position with numpy : 0.0026998519897460938 nb_pixel_total : 76942 time to create 1 rle with old method : 0.08589649200439453 length of segment : 430 time for calcul the mask position with numpy : 0.0006892681121826172 nb_pixel_total : 12916 time to create 1 rle with old method : 0.014551162719726562 length of segment : 122 time for calcul the mask position with numpy : 0.007896900177001953 nb_pixel_total : 86829 time to create 1 rle with old method : 0.09686422348022461 length of segment : 698 time for calcul the mask position with numpy : 0.0037353038787841797 nb_pixel_total : 91369 time to create 1 rle with old method : 0.09674549102783203 length of segment : 533 time for calcul the mask position with numpy : 0.0006442070007324219 nb_pixel_total : 14428 time to create 1 rle with old method : 0.016467809677124023 length of segment : 169 time for calcul the mask position with numpy : 0.0007548332214355469 nb_pixel_total : 36888 time to create 1 rle with old method : 0.040085792541503906 length of segment : 352 time for calcul the mask position with numpy : 0.002950906753540039 nb_pixel_total : 74422 time to create 1 rle with old method : 0.08196330070495605 length of segment : 222 time for calcul the mask position with numpy : 0.0015571117401123047 nb_pixel_total : 26275 time to create 1 rle with old method : 0.02957892417907715 length of segment : 283 time for calcul the mask position with numpy : 0.0010120868682861328 nb_pixel_total : 32798 time to create 1 rle with old method : 0.03656792640686035 length of segment : 182 time for calcul the mask position with numpy : 0.0008153915405273438 nb_pixel_total : 16609 time to create 1 rle with old method : 0.019406795501708984 length of segment : 121 time for calcul the mask position with numpy : 0.0015444755554199219 nb_pixel_total : 42091 time to create 1 rle with old method : 0.046518802642822266 length of segment : 217 time for calcul the mask position with numpy : 0.002307415008544922 nb_pixel_total : 48692 time to create 1 rle with old method : 0.05339384078979492 length of segment : 382 time for calcul the mask position with numpy : 0.0013415813446044922 nb_pixel_total : 40057 time to create 1 rle with old method : 0.04517674446105957 length of segment : 227 time for calcul the mask position with numpy : 0.0023047924041748047 nb_pixel_total : 58137 time to create 1 rle with old method : 0.06362605094909668 length of segment : 360 time for calcul the mask position with numpy : 0.0005993843078613281 nb_pixel_total : 14462 time to create 1 rle with old method : 0.017390012741088867 length of segment : 196 time for calcul the mask position with numpy : 0.0013167858123779297 nb_pixel_total : 57400 time to create 1 rle with old method : 0.0749807357788086 length of segment : 312 time for calcul the mask position with numpy : 0.0016131401062011719 nb_pixel_total : 16814 time to create 1 rle with old method : 0.019535064697265625 length of segment : 596 time for calcul the mask position with numpy : 0.0012202262878417969 nb_pixel_total : 31919 time to create 1 rle with old method : 0.03590273857116699 length of segment : 274 time for calcul the mask position with numpy : 0.0035524368286132812 nb_pixel_total : 84453 time to create 1 rle with old method : 0.10637736320495605 length of segment : 433 time for calcul the mask position with numpy : 0.0017809867858886719 nb_pixel_total : 52579 time to create 1 rle with old method : 0.061890602111816406 length of segment : 251 time for calcul the mask position with numpy : 0.0031366348266601562 nb_pixel_total : 92817 time to create 1 rle with old method : 0.11487817764282227 length of segment : 423 time for calcul the mask position with numpy : 0.0013971328735351562 nb_pixel_total : 47002 time to create 1 rle with old method : 0.05382680892944336 length of segment : 223 time for calcul the mask position with numpy : 0.0014238357543945312 nb_pixel_total : 36457 time to create 1 rle with old method : 0.042862653732299805 length of segment : 205 time for calcul the mask position with numpy : 0.003433704376220703 nb_pixel_total : 88781 time to create 1 rle with old method : 0.10272622108459473 length of segment : 452 time for calcul the mask position with numpy : 0.0018410682678222656 nb_pixel_total : 45424 time to create 1 rle with old method : 0.051261186599731445 length of segment : 314 time for calcul the mask position with numpy : 0.0009353160858154297 nb_pixel_total : 25088 time to create 1 rle with old method : 0.028605222702026367 length of segment : 247 time for calcul the mask position with numpy : 0.0012106895446777344 nb_pixel_total : 34978 time to create 1 rle with old method : 0.04100823402404785 length of segment : 297 time for calcul the mask position with numpy : 0.0012428760528564453 nb_pixel_total : 42495 time to create 1 rle with old method : 0.05003166198730469 length of segment : 230 time for calcul the mask position with numpy : 0.003168344497680664 nb_pixel_total : 60313 time to create 1 rle with old method : 0.07312893867492676 length of segment : 244 time for calcul the mask position with numpy : 0.0008985996246337891 nb_pixel_total : 38520 time to create 1 rle with old method : 0.04515528678894043 length of segment : 278 time for calcul the mask position with numpy : 0.00036716461181640625 nb_pixel_total : 14073 time to create 1 rle with old method : 0.019765138626098633 length of segment : 124 time for calcul the mask position with numpy : 0.00028824806213378906 nb_pixel_total : 8200 time to create 1 rle with old method : 0.009784460067749023 length of segment : 109 time for calcul the mask position with numpy : 0.00069427490234375 nb_pixel_total : 30165 time to create 1 rle with old method : 0.03502178192138672 length of segment : 173 time for calcul the mask position with numpy : 0.0005328655242919922 nb_pixel_total : 19338 time to create 1 rle with old method : 0.02233147621154785 length of segment : 162 time for calcul the mask position with numpy : 0.0011718273162841797 nb_pixel_total : 55468 time to create 1 rle with old method : 0.06309318542480469 length of segment : 324 time for calcul the mask position with numpy : 0.004979372024536133 nb_pixel_total : 185074 time to create 1 rle with new method : 0.012660026550292969 length of segment : 761 time for calcul the mask position with numpy : 0.0022618770599365234 nb_pixel_total : 106679 time to create 1 rle with old method : 0.12502336502075195 length of segment : 365 time for calcul the mask position with numpy : 0.00095367431640625 nb_pixel_total : 36308 time to create 1 rle with old method : 0.03995513916015625 length of segment : 300 time for calcul the mask position with numpy : 0.0018770694732666016 nb_pixel_total : 120756 time to create 1 rle with old method : 0.13059401512145996 length of segment : 501 time for calcul the mask position with numpy : 0.0036306381225585938 nb_pixel_total : 178454 time to create 1 rle with new method : 0.0067331790924072266 length of segment : 340 time for calcul the mask position with numpy : 0.0014069080352783203 nb_pixel_total : 58561 time to create 1 rle with old method : 0.06518864631652832 length of segment : 250 time for calcul the mask position with numpy : 0.00047516822814941406 nb_pixel_total : 16606 time to create 1 rle with old method : 0.018526077270507812 length of segment : 122 time for calcul the mask position with numpy : 0.00249481201171875 nb_pixel_total : 124386 time to create 1 rle with old method : 0.1352534294128418 length of segment : 411 time for calcul the mask position with numpy : 0.006699562072753906 nb_pixel_total : 71686 time to create 1 rle with old method : 0.07844018936157227 length of segment : 316 time for calcul the mask position with numpy : 0.0009152889251708984 nb_pixel_total : 23573 time to create 1 rle with old method : 0.026616573333740234 length of segment : 236 time for calcul the mask position with numpy : 0.0011386871337890625 nb_pixel_total : 39669 time to create 1 rle with old method : 0.04539752006530762 length of segment : 299 time for calcul the mask position with numpy : 0.001627206802368164 nb_pixel_total : 54104 time to create 1 rle with old method : 0.05884909629821777 length of segment : 313 time for calcul the mask position with numpy : 0.0017819404602050781 nb_pixel_total : 77967 time to create 1 rle with old method : 0.08638596534729004 length of segment : 308 time for calcul the mask position with numpy : 0.0023546218872070312 nb_pixel_total : 91688 time to create 1 rle with old method : 0.1004486083984375 length of segment : 397 time for calcul the mask position with numpy : 0.0005156993865966797 nb_pixel_total : 17322 time to create 1 rle with old method : 0.019435405731201172 length of segment : 141 time for calcul the mask position with numpy : 0.0008649826049804688 nb_pixel_total : 33461 time to create 1 rle with old method : 0.03746175765991211 length of segment : 228 time spent for convertir_results : 11.446894645690918 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 103 chid ids of type : 3594 Number RLEs to save : 32763 save missing photos in datou_result : time spend for datou_step_exec : 55.94372367858887 time spend to save output : 2.3949151039123535 total time spend for step 1 : 58.33863878250122 step2:crop_condition Mon Feb 17 17:41:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 5 ! batch 1 Loaded 103 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 82 About to insert : list_path_to_insert length 82 new photo from crops ! About to upload 82 photos upload in portfolio : 3736932 init cache_photo without model_param we have 82 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739810511_174439 we have uploaded 82 photos in the portfolio 3736932 time of upload the photos Elapsed time : 21.361469984054565 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 13 About to insert : list_path_to_insert length 13 new photo from crops ! About to upload 13 photos upload in portfolio : 3736932 init cache_photo without model_param we have 13 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739810536_174439 we have uploaded 13 photos in the portfolio 3736932 time of upload the photos Elapsed time : 3.8337247371673584 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 3736932 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739810541_174439 we have uploaded 3 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.9653446674346924 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 3736932 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739810544_174439 we have uploaded 3 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.8256118297576904 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739810546_174439 we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.6250674724578857 we have finished the crop for the class : pehd begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739810548_174439 we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.6575543880462646 we have finished the crop for the class : pet_fonce delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1338187697, 1338187694, 1338187691, 1338187687, 1338187685] Looping around the photos to save general results len do output : 103 /1338192475Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192476Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192477Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192478Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192479Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192480Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192481Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192482Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192483Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192484Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192485Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192486Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192487Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192488Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192489Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192490Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192491Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192492Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192493Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192494Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192495Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192496Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192497Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192498Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192499Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192500Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192501Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192502Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192503Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192504Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192505Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192506Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192507Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192508Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192509Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192510Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192511Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192512Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192513Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192514Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192515Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192516Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192517Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192518Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192519Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192520Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192521Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192522Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192523Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192524Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192525Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192526Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192527Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192528Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192529Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192530Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192531Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192532Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192533Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192534Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192535Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192536Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192537Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192538Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192539Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192540Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192541Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192542Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192543Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192544Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192545Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192546Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192547Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192548Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192549Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192550Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192551Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192552Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192553Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192554Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192556Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192557Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192559Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192560Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192561Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192562Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192563Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192564Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192565Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192566Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192567Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192568Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192569Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192570Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192571Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192573Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192574Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192575Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192577Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192578Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192579Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192580Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338192584Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2593986') ('3318', '20615073', '1338187697', None, None, None, None, None, '2593986') ('3318', None, None, None, None, None, None, None, '2593986') ('3318', '20615073', '1338187694', None, None, None, None, None, '2593986') ('3318', None, None, None, None, None, None, None, '2593986') ('3318', '20615073', '1338187691', None, None, None, None, None, '2593986') ('3318', None, None, None, None, None, None, None, '2593986') ('3318', '20615073', '1338187687', None, None, None, None, None, '2593986') ('3318', None, None, None, None, None, None, None, '2593986') ('3318', '20615073', '1338187685', None, None, None, None, None, '2593986') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 314 time used for this insertion : 0.02403712272644043 save_final save missing photos in datou_result : time spend for datou_step_exec : 60.51054358482361 time spend to save output : 0.027925968170166016 total time spend for step 2 : 60.538469552993774 step3:rle_unique_nms_with_priority Mon Feb 17 17:42:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 103 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 13 nb_hashtags : 2 time to prepare the origin masks : 5.603684186935425 time for calcul the mask position with numpy : 0.6777276992797852 nb_pixel_total : 7663008 time to create 1 rle with new method : 0.5212039947509766 time for calcul the mask position with numpy : 0.04257488250732422 nb_pixel_total : 53655 time to create 1 rle with old method : 0.06078767776489258 time for calcul the mask position with numpy : 0.0396733283996582 nb_pixel_total : 28115 time to create 1 rle with old method : 0.031317710876464844 time for calcul the mask position with numpy : 0.02771472930908203 nb_pixel_total : 43727 time to create 1 rle with old method : 0.05017900466918945 time for calcul the mask position with numpy : 0.027053356170654297 nb_pixel_total : 18760 time to create 1 rle with old method : 0.02288961410522461 time for calcul the mask position with numpy : 0.029386281967163086 nb_pixel_total : 22228 time to create 1 rle with old method : 0.02479267120361328 time for calcul the mask position with numpy : 0.02693319320678711 nb_pixel_total : 19172 time to create 1 rle with old method : 0.022153615951538086 time for calcul the mask position with numpy : 0.026888132095336914 nb_pixel_total : 51746 time to create 1 rle with old method : 0.05763888359069824 time for calcul the mask position with numpy : 0.027484893798828125 nb_pixel_total : 29156 time to create 1 rle with old method : 0.033240556716918945 time for calcul the mask position with numpy : 0.025484800338745117 nb_pixel_total : 27050 time to create 1 rle with old method : 0.0300900936126709 time for calcul the mask position with numpy : 0.028625011444091797 nb_pixel_total : 140088 time to create 1 rle with old method : 0.15777921676635742 time for calcul the mask position with numpy : 0.030501842498779297 nb_pixel_total : 121094 time to create 1 rle with old method : 0.13657259941101074 time for calcul the mask position with numpy : 0.029530048370361328 nb_pixel_total : 39088 time to create 1 rle with old method : 0.04387044906616211 time for calcul the mask position with numpy : 0.03144240379333496 nb_pixel_total : 37513 time to create 1 rle with old method : 0.057099342346191406 create new chi : 2.367258071899414 time to delete rle : 0.015039205551147461 batch 1 Loaded 27 chid ids of type : 3594 ++++++++++++++++++Number RLEs to save : 10302 TO DO : save crop sub photo not yet done ! save time : 0.5873498916625977 nb_obj : 31 nb_hashtags : 3 time to prepare the origin masks : 4.601618766784668 time for calcul the mask position with numpy : 0.45885753631591797 nb_pixel_total : 6648976 time to create 1 rle with new method : 2.4646830558776855 time for calcul the mask position with numpy : 0.03827810287475586 nb_pixel_total : 24440 time to create 1 rle with old method : 0.029482603073120117 time for calcul the mask position with numpy : 0.03530740737915039 nb_pixel_total : 53112 time to create 1 rle with old method : 0.05846858024597168 time for calcul the mask position with numpy : 0.03513169288635254 nb_pixel_total : 117722 time to create 1 rle with old method : 0.12927675247192383 time for calcul the mask position with numpy : 0.03474545478820801 nb_pixel_total : 47017 time to create 1 rle with old method : 0.052689552307128906 time for calcul the mask position with numpy : 0.0346682071685791 nb_pixel_total : 22509 time to create 1 rle with old method : 0.025527477264404297 time for calcul the mask position with numpy : 0.035033226013183594 nb_pixel_total : 18547 time to create 1 rle with old method : 0.02151203155517578 time for calcul the mask position with numpy : 0.036975860595703125 nb_pixel_total : 40293 time to create 1 rle with old method : 0.0572209358215332 time for calcul the mask position with numpy : 0.03959798812866211 nb_pixel_total : 23850 time to create 1 rle with old method : 0.028596878051757812 time for calcul the mask position with numpy : 0.034485578536987305 nb_pixel_total : 16400 time to create 1 rle with old method : 0.018991947174072266 time for calcul the mask position with numpy : 0.033814191818237305 nb_pixel_total : 32088 time to create 1 rle with old method : 0.03504514694213867 time for calcul the mask position with numpy : 0.04187631607055664 nb_pixel_total : 53030 time to create 1 rle with old method : 0.059009552001953125 time for calcul the mask position with numpy : 0.04018568992614746 nb_pixel_total : 466016 time to create 1 rle with new method : 0.9394941329956055 time for calcul the mask position with numpy : 0.033626556396484375 nb_pixel_total : 32176 time to create 1 rle with old method : 0.03510308265686035 time for calcul the mask position with numpy : 0.03475594520568848 nb_pixel_total : 124097 time to create 1 rle with old method : 0.13389015197753906 time for calcul the mask position with numpy : 0.031772613525390625 nb_pixel_total : 33866 time to create 1 rle with old method : 0.03521466255187988 time for calcul the mask position with numpy : 0.03316617012023926 nb_pixel_total : 45515 time to create 1 rle with old method : 0.04937148094177246 time for calcul the mask position with numpy : 0.03346061706542969 nb_pixel_total : 20491 time to create 1 rle with old method : 0.022921085357666016 time for calcul the mask position with numpy : 0.034380435943603516 nb_pixel_total : 33548 time to create 1 rle with old method : 0.04129481315612793 time for calcul the mask position with numpy : 0.035477638244628906 nb_pixel_total : 79428 time to create 1 rle with old method : 0.08811593055725098 time for calcul the mask position with numpy : 0.03317904472351074 nb_pixel_total : 39091 time to create 1 rle with old method : 0.04716753959655762 time for calcul the mask position with numpy : 0.0335848331451416 nb_pixel_total : 15396 time to create 1 rle with old method : 0.016531944274902344 time for calcul the mask position with numpy : 0.03392648696899414 nb_pixel_total : 10243 time to create 1 rle with old method : 0.013232707977294922 time for calcul the mask position with numpy : 0.035088300704956055 nb_pixel_total : 41611 time to create 1 rle with old method : 0.04826235771179199 time for calcul the mask position with numpy : 0.03392791748046875 nb_pixel_total : 33145 time to create 1 rle with old method : 0.03710603713989258 time for calcul the mask position with numpy : 0.03375053405761719 nb_pixel_total : 51547 time to create 1 rle with old method : 0.05679583549499512 time for calcul the mask position with numpy : 0.033833980560302734 nb_pixel_total : 57707 time to create 1 rle with old method : 0.06195425987243652 time for calcul the mask position with numpy : 0.033200979232788086 nb_pixel_total : 49560 time to create 1 rle with old method : 0.05399823188781738 time for calcul the mask position with numpy : 0.03309130668640137 nb_pixel_total : 11087 time to create 1 rle with old method : 0.011898279190063477 time for calcul the mask position with numpy : 0.03373098373413086 nb_pixel_total : 10839 time to create 1 rle with old method : 0.012252330780029297 time for calcul the mask position with numpy : 0.036072731018066406 nb_pixel_total : 22950 time to create 1 rle with old method : 0.025442838668823242 time for calcul the mask position with numpy : 0.03726482391357422 nb_pixel_total : 18103 time to create 1 rle with old method : 0.020236968994140625 create new chi : 6.342957258224487 time to delete rle : 0.0038836002349853516 batch 1 Loaded 63 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++Number RLEs to save : 21263 TO DO : save crop sub photo not yet done ! save time : 1.1870713233947754 nb_obj : 25 nb_hashtags : 6 time to prepare the origin masks : 7.259792327880859 time for calcul the mask position with numpy : 0.29137635231018066 nb_pixel_total : 7082071 time to create 1 rle with new method : 0.5917246341705322 time for calcul the mask position with numpy : 0.02473592758178711 nb_pixel_total : 16814 time to create 1 rle with old method : 0.018436431884765625 time for calcul the mask position with numpy : 0.023584365844726562 nb_pixel_total : 57400 time to create 1 rle with old method : 0.06103706359863281 time for calcul the mask position with numpy : 0.03881645202636719 nb_pixel_total : 14036 time to create 1 rle with old method : 0.015705108642578125 time for calcul the mask position with numpy : 0.03874635696411133 nb_pixel_total : 58137 time to create 1 rle with old method : 0.06324172019958496 time for calcul the mask position with numpy : 0.03903603553771973 nb_pixel_total : 40057 time to create 1 rle with old method : 0.042194366455078125 time for calcul the mask position with numpy : 0.03815579414367676 nb_pixel_total : 48692 time to create 1 rle with old method : 0.051367759704589844 time for calcul the mask position with numpy : 0.04012489318847656 nb_pixel_total : 42091 time to create 1 rle with old method : 0.04505491256713867 time for calcul the mask position with numpy : 0.04124569892883301 nb_pixel_total : 16609 time to create 1 rle with old method : 0.02110743522644043 time for calcul the mask position with numpy : 0.04195737838745117 nb_pixel_total : 31854 time to create 1 rle with old method : 0.03538250923156738 time for calcul the mask position with numpy : 0.04059290885925293 nb_pixel_total : 26275 time to create 1 rle with old method : 0.02965712547302246 time for calcul the mask position with numpy : 0.038374900817871094 nb_pixel_total : 74422 time to create 1 rle with old method : 0.0931863784790039 time for calcul the mask position with numpy : 0.03279876708984375 nb_pixel_total : 36888 time to create 1 rle with old method : 0.04898428916931152 time for calcul the mask position with numpy : 0.02916860580444336 nb_pixel_total : 14428 time to create 1 rle with old method : 0.02232527732849121 time for calcul the mask position with numpy : 0.026248931884765625 nb_pixel_total : 91369 time to create 1 rle with old method : 0.10023307800292969 time for calcul the mask position with numpy : 0.02568364143371582 nb_pixel_total : 86829 time to create 1 rle with old method : 0.09897613525390625 time for calcul the mask position with numpy : 0.025045394897460938 nb_pixel_total : 12916 time to create 1 rle with old method : 0.014191150665283203 time for calcul the mask position with numpy : 0.028245210647583008 nb_pixel_total : 76942 time to create 1 rle with old method : 0.08453989028930664 time for calcul the mask position with numpy : 0.025603532791137695 nb_pixel_total : 54600 time to create 1 rle with old method : 0.06109476089477539 time for calcul the mask position with numpy : 0.025551557540893555 nb_pixel_total : 56477 time to create 1 rle with old method : 0.06067514419555664 time for calcul the mask position with numpy : 0.024388790130615234 nb_pixel_total : 41125 time to create 1 rle with old method : 0.04569101333618164 time for calcul the mask position with numpy : 0.025372743606567383 nb_pixel_total : 86089 time to create 1 rle with old method : 0.08937501907348633 time for calcul the mask position with numpy : 0.023680686950683594 nb_pixel_total : 35068 time to create 1 rle with old method : 0.03797101974487305 time for calcul the mask position with numpy : 0.0252532958984375 nb_pixel_total : 30543 time to create 1 rle with old method : 0.03235650062561035 time for calcul the mask position with numpy : 0.024857282638549805 nb_pixel_total : 98837 time to create 1 rle with old method : 0.10270905494689941 time for calcul the mask position with numpy : 0.023322105407714844 nb_pixel_total : 63831 time to create 1 rle with old method : 0.0648949146270752 create new chi : 3.031836748123169 time to delete rle : 0.0020134449005126953 batch 1 Loaded 51 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++Number RLEs to save : 18759 TO DO : save crop sub photo not yet done ! save time : 1.0869436264038086 nb_obj : 23 nb_hashtags : 2 time to prepare the origin masks : 5.8157031536102295 time for calcul the mask position with numpy : 0.46940040588378906 nb_pixel_total : 6860375 time to create 1 rle with new method : 0.5199298858642578 time for calcul the mask position with numpy : 0.031751155853271484 nb_pixel_total : 178454 time to create 1 rle with new method : 0.5701026916503906 time for calcul the mask position with numpy : 0.02382516860961914 nb_pixel_total : 120756 time to create 1 rle with old method : 0.12216997146606445 time for calcul the mask position with numpy : 0.023777008056640625 nb_pixel_total : 36308 time to create 1 rle with old method : 0.03844881057739258 time for calcul the mask position with numpy : 0.02431797981262207 nb_pixel_total : 106679 time to create 1 rle with old method : 0.10831427574157715 time for calcul the mask position with numpy : 0.023965120315551758 nb_pixel_total : 185074 time to create 1 rle with new method : 0.4538893699645996 time for calcul the mask position with numpy : 0.02588820457458496 nb_pixel_total : 55468 time to create 1 rle with old method : 0.05989360809326172 time for calcul the mask position with numpy : 0.024218320846557617 nb_pixel_total : 19338 time to create 1 rle with old method : 0.020959138870239258 time for calcul the mask position with numpy : 0.024945497512817383 nb_pixel_total : 30165 time to create 1 rle with old method : 0.0317683219909668 time for calcul the mask position with numpy : 0.024039506912231445 nb_pixel_total : 8200 time to create 1 rle with old method : 0.008996009826660156 time for calcul the mask position with numpy : 0.024483680725097656 nb_pixel_total : 12757 time to create 1 rle with old method : 0.013865947723388672 time for calcul the mask position with numpy : 0.02540731430053711 nb_pixel_total : 38520 time to create 1 rle with old method : 0.041788339614868164 time for calcul the mask position with numpy : 0.024982452392578125 nb_pixel_total : 60313 time to create 1 rle with old method : 0.062212467193603516 time for calcul the mask position with numpy : 0.025084495544433594 nb_pixel_total : 42495 time to create 1 rle with old method : 0.044197797775268555 time for calcul the mask position with numpy : 0.024214506149291992 nb_pixel_total : 34978 time to create 1 rle with old method : 0.0370786190032959 time for calcul the mask position with numpy : 0.024060964584350586 nb_pixel_total : 25088 time to create 1 rle with old method : 0.026125431060791016 time for calcul the mask position with numpy : 0.02427530288696289 nb_pixel_total : 45424 time to create 1 rle with old method : 0.0482480525970459 time for calcul the mask position with numpy : 0.02493453025817871 nb_pixel_total : 88781 time to create 1 rle with old method : 0.09362578392028809 time for calcul the mask position with numpy : 0.03539276123046875 nb_pixel_total : 36457 time to create 1 rle with old method : 0.05700206756591797 time for calcul the mask position with numpy : 0.04234623908996582 nb_pixel_total : 47002 time to create 1 rle with old method : 0.05037522315979004 time for calcul the mask position with numpy : 0.0404965877532959 nb_pixel_total : 92817 time to create 1 rle with old method : 0.10055780410766602 time for calcul the mask position with numpy : 0.04140281677246094 nb_pixel_total : 52579 time to create 1 rle with old method : 0.057105302810668945 time for calcul the mask position with numpy : 0.03372645378112793 nb_pixel_total : 84453 time to create 1 rle with old method : 0.09276580810546875 time for calcul the mask position with numpy : 0.025699615478515625 nb_pixel_total : 31919 time to create 1 rle with old method : 0.0372920036315918 create new chi : 3.9064929485321045 time to delete rle : 0.0019078254699707031 batch 1 Loaded 47 chid ids of type : 3594 +++++++++++++++++++++++++++++++Number RLEs to save : 16173 TO DO : save crop sub photo not yet done ! save time : 1.6634864807128906 nb_obj : 11 nb_hashtags : 4 time to prepare the origin masks : 4.158704996109009 time for calcul the mask position with numpy : 0.4985182285308838 nb_pixel_total : 7685377 time to create 1 rle with new method : 0.7728540897369385 time for calcul the mask position with numpy : 0.04727888107299805 nb_pixel_total : 33461 time to create 1 rle with old method : 0.039078712463378906 time for calcul the mask position with numpy : 0.049245595932006836 nb_pixel_total : 17322 time to create 1 rle with old method : 0.020967960357666016 time for calcul the mask position with numpy : 0.04816699028015137 nb_pixel_total : 91688 time to create 1 rle with old method : 0.10373783111572266 time for calcul the mask position with numpy : 0.04111599922180176 nb_pixel_total : 77967 time to create 1 rle with old method : 0.08828973770141602 time for calcul the mask position with numpy : 0.03730916976928711 nb_pixel_total : 54104 time to create 1 rle with old method : 0.060100555419921875 time for calcul the mask position with numpy : 0.03217649459838867 nb_pixel_total : 39669 time to create 1 rle with old method : 0.04362845420837402 time for calcul the mask position with numpy : 0.024449586868286133 nb_pixel_total : 23573 time to create 1 rle with old method : 0.02590346336364746 time for calcul the mask position with numpy : 0.024685144424438477 nb_pixel_total : 71686 time to create 1 rle with old method : 0.07765913009643555 time for calcul the mask position with numpy : 0.027285099029541016 nb_pixel_total : 124386 time to create 1 rle with old method : 0.13768792152404785 time for calcul the mask position with numpy : 0.02706623077392578 nb_pixel_total : 16606 time to create 1 rle with old method : 0.026920080184936523 time for calcul the mask position with numpy : 0.028847932815551758 nb_pixel_total : 58561 time to create 1 rle with old method : 0.06877803802490234 create new chi : 2.39961576461792 time to delete rle : 0.001995086669921875 batch 1 Loaded 23 chid ids of type : 3594 ++++++++++++++Number RLEs to save : 8202 TO DO : save crop sub photo not yet done ! save time : 0.49431753158569336 map_output_result : {1338187697: (0.0, 'Should be the crop_list due to order', 0.0), 1338187694: (0.0, 'Should be the crop_list due to order', 0.0), 1338187691: (0.0, 'Should be the crop_list due to order', 0.0), 1338187687: (0.0, 'Should be the crop_list due to order', 0.0), 1338187685: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1338187697, 1338187694, 1338187691, 1338187687, 1338187685] Looping around the photos to save general results len do output : 5 /1338187697.Didn't retrieve data . /1338187694.Didn't retrieve data . /1338187691.Didn't retrieve data . /1338187687.Didn't retrieve data . /1338187685.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2593986') ('3318', '20615073', '1338187697', None, None, None, None, None, '2593986') ('3318', None, None, None, None, None, None, None, '2593986') ('3318', '20615073', '1338187694', None, None, None, None, None, '2593986') ('3318', None, None, None, None, None, None, None, '2593986') ('3318', '20615073', '1338187691', None, None, None, None, None, '2593986') ('3318', None, None, None, None, None, None, None, '2593986') ('3318', '20615073', '1338187687', None, None, None, None, None, '2593986') ('3318', None, None, None, None, None, None, None, '2593986') ('3318', '20615073', '1338187685', None, None, None, None, None, '2593986') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 15 time used for this insertion : 0.014457941055297852 save_final save missing photos in datou_result : time spend for datou_step_exec : 51.47898626327515 time spend to save output : 0.014848709106445312 total time spend for step 3 : 51.49383497238159 step4:ventilate_hashtags_in_portfolio Mon Feb 17 17:43:20 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 20615073 get user id for portfolio 20615073 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20615073 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_fonce','carton','autre','background','metal','environnement','papier','pet_clair','mal_croppe','flou','pehd')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20615073 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_fonce','carton','autre','background','metal','environnement','papier','pet_clair','mal_croppe','flou','pehd')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20615073 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_fonce','carton','autre','background','metal','environnement','papier','pet_clair','mal_croppe','flou','pehd')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/20615417,20615418,20615419,20615420,20615421,20615422,20615423,20615424,20615425,20615426,20615427?tags=pet_fonce,carton,autre,background,metal,environnement,papier,pet_clair,mal_croppe,flou,pehd Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1338187697, 1338187694, 1338187691, 1338187687, 1338187685] Looping around the photos to save general results len do output : 1 /20615073. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2593986') ('3318', '20615073', '1338187697', None, None, None, None, None, '2593986') ('3318', None, None, None, None, None, None, None, '2593986') ('3318', '20615073', '1338187694', None, None, None, None, None, '2593986') ('3318', None, None, None, None, None, None, None, '2593986') ('3318', '20615073', '1338187691', None, None, None, None, None, '2593986') ('3318', None, None, None, None, None, None, None, '2593986') ('3318', '20615073', '1338187687', None, None, None, None, None, '2593986') ('3318', None, None, None, None, None, None, None, '2593986') ('3318', '20615073', '1338187685', None, None, None, None, None, '2593986') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.014807939529418945 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.574004888534546 time spend to save output : 0.01504206657409668 total time spend for step 4 : 1.5890469551086426 step5:final Mon Feb 17 17:43:21 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1338187697: ('0.13339585744598764',), 1338187694: ('0.13339585744598764',), 1338187691: ('0.13339585744598764',), 1338187687: ('0.13339585744598764',), 1338187685: ('0.13339585744598764',)} new output for save of step final : {1338187697: ('0.13339585744598764',), 1338187694: ('0.13339585744598764',), 1338187691: ('0.13339585744598764',), 1338187687: ('0.13339585744598764',), 1338187685: ('0.13339585744598764',)} [1338187697, 1338187694, 1338187691, 1338187687, 1338187685] Looping around the photos to save general results len do output : 5 /1338187697.Didn't retrieve data . /1338187694.Didn't retrieve data . /1338187691.Didn't retrieve data . /1338187687.Didn't retrieve data . /1338187685.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2593986') ('3318', '20615073', '1338187697', None, None, None, None, None, '2593986') ('3318', None, None, None, None, None, None, None, '2593986') ('3318', '20615073', '1338187694', None, None, None, None, None, '2593986') ('3318', None, None, None, None, None, None, None, '2593986') ('3318', '20615073', '1338187691', None, None, None, None, None, '2593986') ('3318', None, None, None, None, None, None, None, '2593986') ('3318', '20615073', '1338187687', None, None, None, None, None, '2593986') ('3318', None, None, None, None, None, None, None, '2593986') ('3318', '20615073', '1338187685', None, None, None, None, None, '2593986') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 15 time used for this insertion : 0.012693166732788086 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.10848832130432129 time spend to save output : 0.013013601303100586 total time spend for step 5 : 0.12150192260742188 step6:blur_detection Mon Feb 17 17:43:21 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1739810428_174439_1338187697_a9e9ecf310e69cf591b92f8f9e55674b.jpg resize: (2160, 3840) 1338187697 -6.005790068155613 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c.jpg resize: (2160, 3840) 1338187694 -5.826617282920047 treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3.jpg resize: (2160, 3840) 1338187691 -5.364607508842659 treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022.jpg resize: (2160, 3840) 1338187687 -5.385898429137324 treat image : temp/1739810428_174439_1338187685_4404fbea5faf0698c610fff7d6054687.jpg resize: (2160, 3840) 1338187685 -5.42816729196512 treat image : temp/1739810428_174439_1338187697_a9e9ecf310e69cf591b92f8f9e55674b_rle_crop_3678192358_0.png resize: (266, 330) 1338192475 -3.719029020078699 treat image : temp/1739810428_174439_1338187697_a9e9ecf310e69cf591b92f8f9e55674b_rle_crop_3678192364_0.png resize: (400, 205) 1338192476 -4.320210174292145 treat image : temp/1739810428_174439_1338187697_a9e9ecf310e69cf591b92f8f9e55674b_rle_crop_3678192353_0.png resize: (223, 256) 1338192477 -1.7798865928618368 treat image : temp/1739810428_174439_1338187697_a9e9ecf310e69cf591b92f8f9e55674b_rle_crop_3678192352_0.png resize: (258, 273) 1338192478 -4.227091622118015 treat image : temp/1739810428_174439_1338187697_a9e9ecf310e69cf591b92f8f9e55674b_rle_crop_3678192356_0.png resize: (291, 149) 1338192479 -2.417440243216199 treat image : temp/1739810428_174439_1338187697_a9e9ecf310e69cf591b92f8f9e55674b_rle_crop_3678192355_0.png resize: (563, 412) 1338192480 -3.1605058615354418 treat image : temp/1739810428_174439_1338187697_a9e9ecf310e69cf591b92f8f9e55674b_rle_crop_3678192354_0.png resize: (382, 397) 1338192481 -3.1424322904618234 treat image : temp/1739810428_174439_1338187697_a9e9ecf310e69cf591b92f8f9e55674b_rle_crop_3678192357_0.png resize: (243, 156) 1338192482 -2.9335148516775718 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192395_0.png resize: (993, 887) 1338192483 -4.942605409285888 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192365_0.png resize: (190, 198) 1338192484 -2.666325500237582 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192374_0.png resize: (187, 146) 1338192485 -3.6232322377322013 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192383_0.png resize: (226, 178) 1338192486 -1.5687653966165893 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192386_0.png resize: (111, 189) 1338192487 -2.7251857456902018 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192393_0.png resize: (277, 234) 1338192488 -3.432971638554917 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192377_0.png resize: (187, 264) 1338192489 -2.5672829703818247 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192370_0.png resize: (358, 224) 1338192490 -2.414678005140804 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192371_0.png resize: (491, 387) 1338192491 -4.12359673627386 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192391_0.png resize: (208, 165) 1338192492 -3.057518854917684 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192385_0.png resize: (212, 251) 1338192493 -3.188810173395124 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192381_0.png resize: (361, 276) 1338192494 -3.091503580689445 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192380_0.png resize: (400, 354) 1338192495 -2.874450375245575 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192366_0.png resize: (188, 317) 1338192496 -1.9252821065784593 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192369_0.png resize: (262, 181) 1338192497 -3.8622668968600555 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192367_0.png resize: (315, 158) 1338192498 -3.301422383550433 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192387_0.png resize: (119, 126) 1338192499 -1.7144793521375994 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192389_0.png resize: (436, 246) 1338192500 -3.031640561739274 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192394_0.png resize: (207, 86) 1338192501 -3.5308121795999567 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192372_0.png resize: (293, 191) 1338192502 -3.3871169259129505 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192379_0.png resize: (159, 174) 1338192503 -3.338343978030905 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192375_0.png resize: (207, 338) 1338192504 0.16067169092642705 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192378_0.png resize: (344, 255) 1338192505 -2.5899268488593843 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192392_0.png resize: (140, 174) 1338192506 -2.8054934854760627 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192376_0.png resize: (196, 336) 1338192507 -3.9046731957698158 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192368_0.png resize: (145, 181) 1338192508 -3.1914947170731276 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192390_0.png resize: (150, 102) 1338192509 -2.9195744080192934 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192373_0.png resize: (371, 225) 1338192510 -4.017720629314163 treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192399_0.png resize: (249, 216) 1338192511 -3.5045707702580327 treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192407_0.png resize: (374, 414) 1338192512 -3.784056721287135 treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192397_0.png resize: (340, 362) 1338192513 -3.9730090726396208 treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192402_0.png resize: (312, 408) 1338192514 -2.671062716172934 treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192404_0.png resize: (424, 212) 1338192515 -3.1790071413301204 treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192418_0.png resize: (195, 94) 1338192516 -2.491079375811692 treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192411_0.png resize: (277, 179) 1338192517 -2.914161934892725 treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192415_0.png resize: (363, 221) 1338192518 -2.893324895585867 treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192405_0.png resize: (121, 158) 1338192519 -2.265602063351652 treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192396_0.png resize: (310, 303) 1338192520 -3.1501905383977964 treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192403_0.png resize: (385, 315) 1338192521 -1.524637285761444 treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192414_0.png resize: (213, 275) 1338192522 -1.787375937169799 treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192406_0.png resize: (439, 697) 1338192523 -3.4615786286608548 treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192401_0.png resize: (483, 125) 1338192524 -3.493997586975161 treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192417_0.png resize: (329, 265) 1338192525 -3.3166078026787638 treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192400_0.png resize: (284, 462) 1338192526 -0.9159804319028614 treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192409_0.png resize: (334, 162) 1338192527 -2.6741804650250076 treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192420_0.png resize: (499, 93) 1338192528 -3.476970077444703 treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192439_0.png resize: (619, 458) 1338192529 -3.6687422830161185 treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192443_0.png resize: (324, 707) 1338192530 -3.9689143775177462 treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192431_0.png resize: (228, 249) 1338192531 1.4209461978085602 treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192422_0.png resize: (422, 407) 1338192532 0.5743100759499378 treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192432_0.png resize: (237, 354) 1338192533 -2.9761730683047576 treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192421_0.png resize: (273, 176) 1338192534 -0.6183590511395451 treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192423_0.png resize: (242, 352) 1338192535 -0.059251261544665 treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192430_0.png resize: (294, 183) 1338192536 -3.1152850489616655 treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192442_0.png resize: (496, 372) 1338192537 -1.1758933575564845 treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192429_0.png resize: (161, 217) 1338192538 -3.6268120393126164 treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192437_0.png resize: (162, 167) 1338192539 -1.963295543165204 treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192441_0.png resize: (281, 256) 1338192540 -2.8634772512714193 treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192427_0.png resize: (380, 357) 1338192541 0.31907686595075907 treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192440_0.png resize: (315, 438) 1338192542 -4.099565378921118 treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192428_0.png resize: (289, 219) 1338192543 -3.246148244010526 treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192424_0.png resize: (419, 284) 1338192544 -1.620028439279533 treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192434_0.png resize: (117, 186) 1338192545 -2.2296554509104585 treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192433_0.png resize: (235, 211) 1338192546 -1.964171616664389 treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192436_0.png resize: (168, 242) 1338192547 -2.62144970736462 treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192435_0.png resize: (109, 93) 1338192548 -2.0583426864704633 treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192426_0.png resize: (203, 309) 1338192549 -3.7428696650340765 treat image : temp/1739810428_174439_1338187685_4404fbea5faf0698c610fff7d6054687_rle_crop_3678192445_0.png resize: (122, 173) 1338192550 -3.3728841633889863 treat image : temp/1739810428_174439_1338187685_4404fbea5faf0698c610fff7d6054687_rle_crop_3678192450_0.png resize: (313, 252) 1338192551 -3.0322729499255803 treat image : temp/1739810428_174439_1338187685_4404fbea5faf0698c610fff7d6054687_rle_crop_3678192444_0.png resize: (250, 329) 1338192552 -2.962928244949066 treat image : temp/1739810428_174439_1338187685_4404fbea5faf0698c610fff7d6054687_rle_crop_3678192446_0.png resize: (350, 498) 1338192553 -4.186095412409553 treat image : temp/1739810428_174439_1338187685_4404fbea5faf0698c610fff7d6054687_rle_crop_3678192454_0.png resize: (228, 306) 1338192554 -2.925422967318872 treat image : temp/1739810428_174439_1338187685_4404fbea5faf0698c610fff7d6054687_rle_crop_3678192451_0.png resize: (260, 413) 1338192556 -3.366297126938021 treat image : temp/1739810428_174439_1338187685_4404fbea5faf0698c610fff7d6054687_rle_crop_3678192447_0.png resize: (312, 355) 1338192557 -3.0964223815529137 treat image : temp/1739810428_174439_1338187697_a9e9ecf310e69cf591b92f8f9e55674b_rle_crop_3678192363_0.png resize: (185, 208) 1338192559 -3.638134975717122 treat image : temp/1739810428_174439_1338187697_a9e9ecf310e69cf591b92f8f9e55674b_rle_crop_3678192362_0.png resize: (346, 213) 1338192560 -3.32536155604066 treat image : temp/1739810428_174439_1338187697_a9e9ecf310e69cf591b92f8f9e55674b_rle_crop_3678192361_0.png resize: (199, 119) 1338192561 -2.924506743348402 treat image : temp/1739810428_174439_1338187697_a9e9ecf310e69cf591b92f8f9e55674b_rle_crop_3678192360_0.png resize: (164, 205) 1338192562 -2.659926409543606 treat image : temp/1739810428_174439_1338187697_a9e9ecf310e69cf591b92f8f9e55674b_rle_crop_3678192359_0.png resize: (196, 116) 1338192563 -3.733705033831989 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192382_0.png resize: (238, 620) 1338192564 -2.6409306981392913 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192388_0.png resize: (296, 222) 1338192565 -3.6650782160902486 treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192398_0.png resize: (244, 201) 1338192566 -3.6743933165383735 treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192408_0.png resize: (169, 113) 1338192567 -2.8723076466994804 treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192438_0.png resize: (322, 215) 1338192568 -4.047275391904667 treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192425_0.png resize: (219, 295) 1338192569 -3.4851202087867725 treat image : temp/1739810428_174439_1338187685_4404fbea5faf0698c610fff7d6054687_rle_crop_3678192449_0.png resize: (299, 188) 1338192570 -3.567342047716397 treat image : temp/1739810428_174439_1338187685_4404fbea5faf0698c610fff7d6054687_rle_crop_3678192452_0.png resize: (335, 377) 1338192571 -4.358706268656985 treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192412_0.png resize: (181, 263) 1338192573 -3.51518924995201 treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192410_0.png resize: (222, 425) 1338192574 -3.227892559484904 treat image : temp/1739810428_174439_1338187685_4404fbea5faf0698c610fff7d6054687_rle_crop_3678192448_0.png resize: (230, 151) 1338192575 -2.1148526104937773 treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192384_0.png resize: (388, 189) 1338192577 -2.1808119799699726 treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192416_0.png resize: (227, 218) 1338192578 -1.3048102150910021 treat image : temp/1739810428_174439_1338187685_4404fbea5faf0698c610fff7d6054687_rle_crop_3678192453_0.png resize: (141, 150) 1338192579 -2.4297337660217475 treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192413_0.png resize: (117, 184) 1338192580 -1.4151573012968035 treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192419_0.png resize: (294, 236) 1338192584 -3.5018981193017864 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 108 time used for this insertion : 0.01723504066467285 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 108 time used for this insertion : 0.029063701629638672 save missing photos in datou_result : time spend for datou_step_exec : 22.159765481948853 time spend to save output : 0.0524289608001709 total time spend for step 6 : 22.212194442749023 step7:brightness Mon Feb 17 17:43:44 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1739810428_174439_1338187697_a9e9ecf310e69cf591b92f8f9e55674b.jpg treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c.jpg treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3.jpg treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022.jpg treat image : temp/1739810428_174439_1338187685_4404fbea5faf0698c610fff7d6054687.jpg treat image : temp/1739810428_174439_1338187697_a9e9ecf310e69cf591b92f8f9e55674b_rle_crop_3678192358_0.png treat image : temp/1739810428_174439_1338187697_a9e9ecf310e69cf591b92f8f9e55674b_rle_crop_3678192364_0.png treat image : temp/1739810428_174439_1338187697_a9e9ecf310e69cf591b92f8f9e55674b_rle_crop_3678192353_0.png treat image : temp/1739810428_174439_1338187697_a9e9ecf310e69cf591b92f8f9e55674b_rle_crop_3678192352_0.png treat image : temp/1739810428_174439_1338187697_a9e9ecf310e69cf591b92f8f9e55674b_rle_crop_3678192356_0.png treat image : temp/1739810428_174439_1338187697_a9e9ecf310e69cf591b92f8f9e55674b_rle_crop_3678192355_0.png treat image : temp/1739810428_174439_1338187697_a9e9ecf310e69cf591b92f8f9e55674b_rle_crop_3678192354_0.png treat image : temp/1739810428_174439_1338187697_a9e9ecf310e69cf591b92f8f9e55674b_rle_crop_3678192357_0.png treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192395_0.png treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192365_0.png treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192374_0.png treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192383_0.png treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192386_0.png treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192393_0.png treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192377_0.png treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192370_0.png treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192371_0.png treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192391_0.png treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192385_0.png treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192381_0.png treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192380_0.png treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192366_0.png treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192369_0.png treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192367_0.png treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192387_0.png treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192389_0.png treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192394_0.png treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192372_0.png treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192379_0.png treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192375_0.png treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192378_0.png treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192392_0.png treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192376_0.png treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192368_0.png treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192390_0.png treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192373_0.png treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192399_0.png treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192407_0.png treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192397_0.png treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192402_0.png treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192404_0.png treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192418_0.png treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192411_0.png treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192415_0.png treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192405_0.png treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192396_0.png treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192403_0.png treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192414_0.png treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192406_0.png treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192401_0.png treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192417_0.png treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192400_0.png treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192409_0.png treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192420_0.png treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192439_0.png treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192443_0.png treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192431_0.png treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192422_0.png treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192432_0.png treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192421_0.png treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192423_0.png treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192430_0.png treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192442_0.png treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192429_0.png treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192437_0.png treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192441_0.png treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192427_0.png treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192440_0.png treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192428_0.png treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192424_0.png treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192434_0.png treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192433_0.png treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192436_0.png treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192435_0.png treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192426_0.png treat image : temp/1739810428_174439_1338187685_4404fbea5faf0698c610fff7d6054687_rle_crop_3678192445_0.png treat image : temp/1739810428_174439_1338187685_4404fbea5faf0698c610fff7d6054687_rle_crop_3678192450_0.png treat image : temp/1739810428_174439_1338187685_4404fbea5faf0698c610fff7d6054687_rle_crop_3678192444_0.png treat image : temp/1739810428_174439_1338187685_4404fbea5faf0698c610fff7d6054687_rle_crop_3678192446_0.png treat image : temp/1739810428_174439_1338187685_4404fbea5faf0698c610fff7d6054687_rle_crop_3678192454_0.png treat image : temp/1739810428_174439_1338187685_4404fbea5faf0698c610fff7d6054687_rle_crop_3678192451_0.png treat image : temp/1739810428_174439_1338187685_4404fbea5faf0698c610fff7d6054687_rle_crop_3678192447_0.png treat image : temp/1739810428_174439_1338187697_a9e9ecf310e69cf591b92f8f9e55674b_rle_crop_3678192363_0.png treat image : temp/1739810428_174439_1338187697_a9e9ecf310e69cf591b92f8f9e55674b_rle_crop_3678192362_0.png treat image : temp/1739810428_174439_1338187697_a9e9ecf310e69cf591b92f8f9e55674b_rle_crop_3678192361_0.png treat image : temp/1739810428_174439_1338187697_a9e9ecf310e69cf591b92f8f9e55674b_rle_crop_3678192360_0.png treat image : temp/1739810428_174439_1338187697_a9e9ecf310e69cf591b92f8f9e55674b_rle_crop_3678192359_0.png treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192382_0.png treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192388_0.png treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192398_0.png treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192408_0.png treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192438_0.png treat image : temp/1739810428_174439_1338187687_645a0d808ddf0d1087c8774dfb9dd022_rle_crop_3678192425_0.png treat image : temp/1739810428_174439_1338187685_4404fbea5faf0698c610fff7d6054687_rle_crop_3678192449_0.png treat image : temp/1739810428_174439_1338187685_4404fbea5faf0698c610fff7d6054687_rle_crop_3678192452_0.png treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192412_0.png treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192410_0.png treat image : temp/1739810428_174439_1338187685_4404fbea5faf0698c610fff7d6054687_rle_crop_3678192448_0.png treat image : temp/1739810428_174439_1338187694_eda833477a2b4fd1e091cbfe1afb744c_rle_crop_3678192384_0.png treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192416_0.png treat image : temp/1739810428_174439_1338187685_4404fbea5faf0698c610fff7d6054687_rle_crop_3678192453_0.png treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192413_0.png treat image : temp/1739810428_174439_1338187691_4977c0bbfb67c3ed4f520af0037f66c3_rle_crop_3678192419_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 108 time used for this insertion : 0.01701498031616211 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 108 time used for this insertion : 0.027107954025268555 save missing photos in datou_result : time spend for datou_step_exec : 6.068380355834961 time spend to save output : 0.04943990707397461 total time spend for step 7 : 6.1178202629089355 step8:velours_tree Mon Feb 17 17:43:50 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.09198355674743652 time spend to save output : 5.3882598876953125e-05 total time spend for step 8 : 0.09203743934631348 step9:send_mail_cod Mon Feb 17 17:43:50 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P20615073_17-02-2025_17_43_50.pdf 20615417 change filename to text .imagette206154171739810630 20615418 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette206154181739810630 20615419 change filename to text .change filename to text .change filename to text .imagette206154191739810631 20615420 imagette206154201739810631 20615421 imagette206154211739810631 20615423 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette206154231739810631 20615424 change filename to text .change filename to text .change filename to text .imagette206154241739810632 20615425 imagette206154251739810633 20615426 imagette206154261739810633 20615427 change filename to text .imagette206154271739810633 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20615073 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20615417,20615418,20615419,20615420,20615421,20615422,20615423,20615424,20615425,20615426,20615427?tags=pet_fonce,carton,autre,background,metal,environnement,papier,pet_clair,mal_croppe,flou,pehd args[1338187697] : ((1338187697, -6.005790068155613, 492609224), (1338187697, 0.3174815215220938, 2107752395), '0.13339585744598764') apple ((1338187697, -6.005790068155613, 492609224), (1338187697, 0.3174815215220938, 2107752395), '0.13339585744598764') We are sending mail with results at report@fotonower.com args[1338187694] : ((1338187694, -5.826617282920047, 492609224), (1338187694, 0.19893491113476336, 2107752395), '0.13339585744598764') apple ((1338187694, -5.826617282920047, 492609224), (1338187694, 0.19893491113476336, 2107752395), '0.13339585744598764') We are sending mail with results at report@fotonower.com args[1338187691] : ((1338187691, -5.364607508842659, 492609224), (1338187691, 0.29767965699737065, 2107752395), '0.13339585744598764') apple ((1338187691, -5.364607508842659, 492609224), (1338187691, 0.29767965699737065, 2107752395), '0.13339585744598764') We are sending mail with results at report@fotonower.com args[1338187687] : ((1338187687, -5.385898429137324, 492609224), (1338187687, 0.7643515383716666, 2107752395), '0.13339585744598764') apple ((1338187687, -5.385898429137324, 492609224), (1338187687, 0.7643515383716666, 2107752395), '0.13339585744598764') We are sending mail with results at report@fotonower.com args[1338187685] : ((1338187685, -5.42816729196512, 492609224), (1338187685, -0.27675308777971713, 496442774), '0.13339585744598764') apple ((1338187685, -5.42816729196512, 492609224), (1338187685, -0.27675308777971713, 496442774), '0.13339585744598764') We are sending mail with results at report@fotonower.com refus_total : 0.13339585744598764 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20615073 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20615073_17-02-2025_17_43_50.pdf results_Auto_P20615073_17-02-2025_17_43_50.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20615073_17-02-2025_17_43_50.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','20615073','results_Auto_P20615073_17-02-2025_17_43_50.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20615073_17-02-2025_17_43_50.pdf','pdf','','0.48','0.13339585744598764') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/20615073

https://www.fotonower.com/image?json=false&list_photos_id=1338187697
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1338187694
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1338187691
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1338187687
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1338187685
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 13.34%
Veuillez trouver les photos des contaminants.

exemples de contaminants: pet_fonce: https://www.fotonower.com/view/20615417?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/20615418?limit=200
exemples de contaminants: autre: https://www.fotonower.com/view/20615419?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/20615423?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/20615424?limit=200
exemples de contaminants: pehd: https://www.fotonower.com/view/20615427?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20615073_17-02-2025_17_43_50.pdf.

Lien vers velours :https://www.fotonower.com/velours/20615417,20615418,20615419,20615420,20615421,20615422,20615423,20615424,20615425,20615426,20615427?tags=pet_fonce,carton,autre,background,metal,environnement,papier,pet_clair,mal_croppe,flou,pehd.


L'équipe Fotonower 202 b'' Server: nginx Date: Mon, 17 Feb 2025 16:43:55 GMT Content-Length: 0 Connection: close X-Message-Id: frip4qJ0SCiAQbIiMnSGBQ Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1338187697, 1338187694, 1338187691, 1338187687, 1338187685] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2593986') ('3318', '20615073', '1338187697', None, None, None, None, None, '2593986') ('3318', None, None, None, None, None, None, None, '2593986') ('3318', '20615073', '1338187694', None, None, None, None, None, '2593986') ('3318', None, None, None, None, None, None, None, '2593986') ('3318', '20615073', '1338187691', None, None, None, None, None, '2593986') ('3318', None, None, None, None, None, None, None, '2593986') ('3318', '20615073', '1338187687', None, None, None, None, None, '2593986') ('3318', None, None, None, None, None, None, None, '2593986') ('3318', '20615073', '1338187685', None, None, None, None, None, '2593986') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 5 time used for this insertion : 0.014191627502441406 save_final save missing photos in datou_result : time spend for datou_step_exec : 5.224869966506958 time spend to save output : 0.01435542106628418 total time spend for step 9 : 5.239225387573242 step10:split_time_score Mon Feb 17 17:43:55 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('17', 5),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 17022025 20615073 Nombre de photos uploadées : 5 / 23040 (0%) 17022025 20615073 Nombre de photos taguées (types de déchets): 0 / 5 (0%) 17022025 20615073 Nombre de photos taguées (volume) : 0 / 5 (0%) elapsed_time : load_data_split_time_score 1.9073486328125e-06 elapsed_time : order_list_meta_photo_and_scores 6.198883056640625e-06 ????? elapsed_time : fill_and_build_computed_from_old_data 0.00038313865661621094 elapsed_time : insert_dashboard_record_day_entry 0.02467632293701172 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.03486663619902253 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20596788_17-02-2025_07_46_04.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20596788 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20596788 AND mptpi.`type`=3726 To do Qualite : 0.011089048529427666 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20598619_17-02-2025_08_50_32.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20598619 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20598619 AND mptpi.`type`=3726 To do Qualite : 0.021268959728253434 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20600234_17-02-2025_11_28_13.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20600234 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20600234 AND mptpi.`type`=3726 To do Qualite : 0.024100637115361326 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20607521_17-02-2025_14_15_07.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20607521 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20607521 AND mptpi.`type`=3726 To do Qualite : 0.03400291459679668 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20607523_17-02-2025_12_55_36.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20607523 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20607523 AND mptpi.`type`=3726 To do Qualite : 0.026363027018224763 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20609889_17-02-2025_15_49_53.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20609889 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20609889 AND mptpi.`type`=3726 To do Qualite : 0.06403087615073508 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20612085_17-02-2025_16_37_15.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20612085 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20612085 AND mptpi.`type`=3726 To do Qualite : 0.13339585744598764 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20615073_17-02-2025_17_43_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20615073 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20615073 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'17022025': {'nb_upload': 5, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1338187697, 1338187694, 1338187691, 1338187687, 1338187685] Looping around the photos to save general results len do output : 1 /20615073Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2593986') ('3318', '20615073', '1338187697', None, None, None, None, None, '2593986') ('3318', None, None, None, None, None, None, None, '2593986') ('3318', '20615073', '1338187694', None, None, None, None, None, '2593986') ('3318', None, None, None, None, None, None, None, '2593986') ('3318', '20615073', '1338187691', None, None, None, None, None, '2593986') ('3318', None, None, None, None, None, None, None, '2593986') ('3318', '20615073', '1338187687', None, None, None, None, None, '2593986') ('3318', None, None, None, None, None, None, None, '2593986') ('3318', '20615073', '1338187685', None, None, None, None, None, '2593986') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.01201176643371582 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.6595642566680908 time spend to save output : 0.012153387069702148 total time spend for step 10 : 0.671717643737793 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 5 set_done_treatment 102.90user 67.38system 3:29.97elapsed 81%CPU (0avgtext+0avgdata 4274380maxresident)k 604648inputs+90312outputs (9813major+9585135minor)pagefaults 0swaps