python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3182161 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['2740212'] with mtr_portfolio_ids : ['22230622'] and first list_photo_ids : [] new path : /proc/3182161/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 7 ; length of list_pids : 7 ; length of list_args : 7 time to download the photos : 1.1635420322418213 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Thu Apr 10 16:10:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10219 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-04-10 16:10:34.034546: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-04-10 16:10:34.059142: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-04-10 16:10:34.061212: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f91b4000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-04-10 16:10:34.061259: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-04-10 16:10:34.097286: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-04-10 16:10:34.234101: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x2c574bd0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-04-10 16:10:34.234157: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-04-10 16:10:34.235312: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-04-10 16:10:34.235674: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-04-10 16:10:34.237943: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-04-10 16:10:34.240045: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-04-10 16:10:34.243878: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-04-10 16:10:34.246416: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-04-10 16:10:34.247777: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-04-10 16:10:34.253046: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-04-10 16:10:34.254828: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-04-10 16:10:34.254952: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-04-10 16:10:34.255924: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-04-10 16:10:34.255943: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-04-10 16:10:34.255953: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-04-10 16:10:34.258191: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9463 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-04-10 16:10:34.524874: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-04-10 16:10:34.524991: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-04-10 16:10:34.525011: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-04-10 16:10:34.525030: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-04-10 16:10:34.525048: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-04-10 16:10:34.525065: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-04-10 16:10:34.525083: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-04-10 16:10:34.525101: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-04-10 16:10:34.526609: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-04-10 16:10:34.527956: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-04-10 16:10:34.527994: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-04-10 16:10:34.528013: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-04-10 16:10:34.528030: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-04-10 16:10:34.528047: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-04-10 16:10:34.528064: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-04-10 16:10:34.528080: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-04-10 16:10:34.528097: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-04-10 16:10:34.529598: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-04-10 16:10:34.529637: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-04-10 16:10:34.529647: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-04-10 16:10:34.529657: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-04-10 16:10:34.531251: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9463 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-04-10 16:10:46.781143: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-04-10 16:10:47.001393: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 7 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 148.35000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 13 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 44 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 146.85000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 45 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 47 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 45 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.60000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 31 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 36 Detection mask done ! Trying to reset tf kernel 3182817 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1308 tf kernel not reseted sub process len(results) : 7 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 7 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10213 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0022602081298828125 nb_pixel_total : 78538 time to create 1 rle with old method : 0.09306597709655762 length of segment : 335 time for calcul the mask position with numpy : 0.001794576644897461 nb_pixel_total : 78934 time to create 1 rle with old method : 0.09147524833679199 length of segment : 362 time for calcul the mask position with numpy : 0.0005326271057128906 nb_pixel_total : 25436 time to create 1 rle with old method : 0.030100584030151367 length of segment : 206 time for calcul the mask position with numpy : 0.00029015541076660156 nb_pixel_total : 20321 time to create 1 rle with old method : 0.02411484718322754 length of segment : 141 time for calcul the mask position with numpy : 0.003536701202392578 nb_pixel_total : 156479 time to create 1 rle with new method : 0.010593652725219727 length of segment : 560 time for calcul the mask position with numpy : 0.00026726722717285156 nb_pixel_total : 15644 time to create 1 rle with old method : 0.018499135971069336 length of segment : 141 time for calcul the mask position with numpy : 0.0012149810791015625 nb_pixel_total : 90332 time to create 1 rle with old method : 0.10539770126342773 length of segment : 548 time for calcul the mask position with numpy : 0.0018663406372070312 nb_pixel_total : 109151 time to create 1 rle with old method : 0.13787412643432617 length of segment : 469 time for calcul the mask position with numpy : 0.0003230571746826172 nb_pixel_total : 14062 time to create 1 rle with old method : 0.01749706268310547 length of segment : 101 time for calcul the mask position with numpy : 0.00576472282409668 nb_pixel_total : 211948 time to create 1 rle with new method : 0.022151708602905273 length of segment : 640 time for calcul the mask position with numpy : 0.0005440711975097656 nb_pixel_total : 28927 time to create 1 rle with old method : 0.03402447700500488 length of segment : 234 time for calcul the mask position with numpy : 0.0006463527679443359 nb_pixel_total : 44517 time to create 1 rle with old method : 0.051749229431152344 length of segment : 213 time for calcul the mask position with numpy : 0.0018727779388427734 nb_pixel_total : 113194 time to create 1 rle with old method : 0.13271164894104004 length of segment : 431 time for calcul the mask position with numpy : 0.000827789306640625 nb_pixel_total : 42592 time to create 1 rle with old method : 0.05051398277282715 length of segment : 226 time for calcul the mask position with numpy : 0.0004341602325439453 nb_pixel_total : 17827 time to create 1 rle with old method : 0.0211794376373291 length of segment : 183 time for calcul the mask position with numpy : 0.0021545886993408203 nb_pixel_total : 131687 time to create 1 rle with old method : 0.15375542640686035 length of segment : 437 time for calcul the mask position with numpy : 0.0007522106170654297 nb_pixel_total : 36757 time to create 1 rle with old method : 0.04639744758605957 length of segment : 244 time for calcul the mask position with numpy : 0.0001575946807861328 nb_pixel_total : 3953 time to create 1 rle with old method : 0.00507807731628418 length of segment : 48 time for calcul the mask position with numpy : 0.0002551078796386719 nb_pixel_total : 11193 time to create 1 rle with old method : 0.01368570327758789 length of segment : 147 time for calcul the mask position with numpy : 0.00034689903259277344 nb_pixel_total : 15068 time to create 1 rle with old method : 0.018584251403808594 length of segment : 120 time for calcul the mask position with numpy : 0.0009796619415283203 nb_pixel_total : 44576 time to create 1 rle with old method : 0.053363800048828125 length of segment : 271 time for calcul the mask position with numpy : 0.001432657241821289 nb_pixel_total : 80484 time to create 1 rle with old method : 0.09095001220703125 length of segment : 328 time for calcul the mask position with numpy : 0.00042247772216796875 nb_pixel_total : 23379 time to create 1 rle with old method : 0.027459383010864258 length of segment : 165 time for calcul the mask position with numpy : 0.0007200241088867188 nb_pixel_total : 37384 time to create 1 rle with old method : 0.05709362030029297 length of segment : 187 time for calcul the mask position with numpy : 0.0003914833068847656 nb_pixel_total : 15622 time to create 1 rle with old method : 0.01891350746154785 length of segment : 205 time for calcul the mask position with numpy : 0.00026702880859375 nb_pixel_total : 14512 time to create 1 rle with old method : 0.016600847244262695 length of segment : 161 time for calcul the mask position with numpy : 0.00047278404235839844 nb_pixel_total : 26438 time to create 1 rle with old method : 0.030485868453979492 length of segment : 205 time for calcul the mask position with numpy : 0.00019502639770507812 nb_pixel_total : 7291 time to create 1 rle with old method : 0.008966922760009766 length of segment : 127 time for calcul the mask position with numpy : 0.0005125999450683594 nb_pixel_total : 29978 time to create 1 rle with old method : 0.03591012954711914 length of segment : 175 time for calcul the mask position with numpy : 0.0008242130279541016 nb_pixel_total : 42867 time to create 1 rle with old method : 0.04882502555847168 length of segment : 410 time for calcul the mask position with numpy : 0.00031375885009765625 nb_pixel_total : 19591 time to create 1 rle with old method : 0.02273082733154297 length of segment : 112 time for calcul the mask position with numpy : 0.0008740425109863281 nb_pixel_total : 53813 time to create 1 rle with old method : 0.06315183639526367 length of segment : 217 time for calcul the mask position with numpy : 0.00026488304138183594 nb_pixel_total : 12693 time to create 1 rle with old method : 0.014935016632080078 length of segment : 149 time for calcul the mask position with numpy : 0.0002808570861816406 nb_pixel_total : 11493 time to create 1 rle with old method : 0.013647794723510742 length of segment : 155 time for calcul the mask position with numpy : 0.0003466606140136719 nb_pixel_total : 18118 time to create 1 rle with old method : 0.02121138572692871 length of segment : 202 time for calcul the mask position with numpy : 0.00022268295288085938 nb_pixel_total : 10905 time to create 1 rle with old method : 0.012989997863769531 length of segment : 109 time for calcul the mask position with numpy : 0.0005071163177490234 nb_pixel_total : 28935 time to create 1 rle with old method : 0.03329873085021973 length of segment : 318 time for calcul the mask position with numpy : 0.0004100799560546875 nb_pixel_total : 22615 time to create 1 rle with old method : 0.025859832763671875 length of segment : 162 time for calcul the mask position with numpy : 0.0001842975616455078 nb_pixel_total : 6684 time to create 1 rle with old method : 0.007969141006469727 length of segment : 148 time for calcul the mask position with numpy : 0.0005583763122558594 nb_pixel_total : 31888 time to create 1 rle with old method : 0.03696441650390625 length of segment : 262 time for calcul the mask position with numpy : 0.0013544559478759766 nb_pixel_total : 75949 time to create 1 rle with old method : 0.0870053768157959 length of segment : 357 time for calcul the mask position with numpy : 0.0004329681396484375 nb_pixel_total : 22120 time to create 1 rle with old method : 0.02544379234313965 length of segment : 315 time for calcul the mask position with numpy : 0.0015766620635986328 nb_pixel_total : 92807 time to create 1 rle with old method : 0.10918521881103516 length of segment : 713 time for calcul the mask position with numpy : 0.0011374950408935547 nb_pixel_total : 69163 time to create 1 rle with old method : 0.07767224311828613 length of segment : 347 time for calcul the mask position with numpy : 0.0007481575012207031 nb_pixel_total : 31993 time to create 1 rle with old method : 0.03624081611633301 length of segment : 211 time for calcul the mask position with numpy : 0.0001952648162841797 nb_pixel_total : 11260 time to create 1 rle with old method : 0.013607978820800781 length of segment : 101 time for calcul the mask position with numpy : 0.0002422332763671875 nb_pixel_total : 10770 time to create 1 rle with old method : 0.012861490249633789 length of segment : 136 time for calcul the mask position with numpy : 0.0009739398956298828 nb_pixel_total : 34497 time to create 1 rle with old method : 0.039487361907958984 length of segment : 293 time for calcul the mask position with numpy : 0.00022149085998535156 nb_pixel_total : 8473 time to create 1 rle with old method : 0.010151386260986328 length of segment : 139 time for calcul the mask position with numpy : 0.0008389949798583984 nb_pixel_total : 20048 time to create 1 rle with old method : 0.023099899291992188 length of segment : 204 time for calcul the mask position with numpy : 0.0008969306945800781 nb_pixel_total : 20127 time to create 1 rle with old method : 0.022698640823364258 length of segment : 269 time for calcul the mask position with numpy : 0.0004165172576904297 nb_pixel_total : 9893 time to create 1 rle with old method : 0.01131129264831543 length of segment : 79 time for calcul the mask position with numpy : 0.00039887428283691406 nb_pixel_total : 5781 time to create 1 rle with old method : 0.006803035736083984 length of segment : 112 time for calcul the mask position with numpy : 0.0026640892028808594 nb_pixel_total : 64735 time to create 1 rle with old method : 0.07312345504760742 length of segment : 363 time for calcul the mask position with numpy : 0.006964206695556641 nb_pixel_total : 198358 time to create 1 rle with new method : 0.011943340301513672 length of segment : 574 time for calcul the mask position with numpy : 0.001756429672241211 nb_pixel_total : 47832 time to create 1 rle with old method : 0.054350852966308594 length of segment : 303 time for calcul the mask position with numpy : 0.0027282238006591797 nb_pixel_total : 73385 time to create 1 rle with old method : 0.09905457496643066 length of segment : 324 time for calcul the mask position with numpy : 0.020034074783325195 nb_pixel_total : 335071 time to create 1 rle with new method : 0.6636903285980225 length of segment : 931 time for calcul the mask position with numpy : 0.0005848407745361328 nb_pixel_total : 15461 time to create 1 rle with old method : 0.018804073333740234 length of segment : 119 time for calcul the mask position with numpy : 0.0007522106170654297 nb_pixel_total : 23802 time to create 1 rle with old method : 0.028687238693237305 length of segment : 207 time for calcul the mask position with numpy : 0.0003857612609863281 nb_pixel_total : 13988 time to create 1 rle with old method : 0.016455888748168945 length of segment : 168 time for calcul the mask position with numpy : 0.0012900829315185547 nb_pixel_total : 69084 time to create 1 rle with old method : 0.07810354232788086 length of segment : 340 time for calcul the mask position with numpy : 0.0014538764953613281 nb_pixel_total : 38514 time to create 1 rle with old method : 0.0437161922454834 length of segment : 270 time for calcul the mask position with numpy : 0.00024175643920898438 nb_pixel_total : 15394 time to create 1 rle with old method : 0.017102718353271484 length of segment : 93 time for calcul the mask position with numpy : 0.0003559589385986328 nb_pixel_total : 13110 time to create 1 rle with old method : 0.015215873718261719 length of segment : 160 time for calcul the mask position with numpy : 0.00022530555725097656 nb_pixel_total : 5628 time to create 1 rle with old method : 0.006325483322143555 length of segment : 128 time for calcul the mask position with numpy : 7.987022399902344e-05 nb_pixel_total : 3009 time to create 1 rle with old method : 0.003419637680053711 length of segment : 104 time for calcul the mask position with numpy : 0.0015091896057128906 nb_pixel_total : 46087 time to create 1 rle with old method : 0.05203986167907715 length of segment : 325 time for calcul the mask position with numpy : 0.0002887248992919922 nb_pixel_total : 6750 time to create 1 rle with old method : 0.008018970489501953 length of segment : 87 time for calcul the mask position with numpy : 0.0037717819213867188 nb_pixel_total : 110641 time to create 1 rle with old method : 0.12230896949768066 length of segment : 411 time for calcul the mask position with numpy : 0.0005078315734863281 nb_pixel_total : 12499 time to create 1 rle with old method : 0.014216899871826172 length of segment : 117 time for calcul the mask position with numpy : 0.0010552406311035156 nb_pixel_total : 15725 time to create 1 rle with old method : 0.017640352249145508 length of segment : 242 time for calcul the mask position with numpy : 0.001277923583984375 nb_pixel_total : 27695 time to create 1 rle with old method : 0.030687332153320312 length of segment : 200 time for calcul the mask position with numpy : 0.0018007755279541016 nb_pixel_total : 53050 time to create 1 rle with old method : 0.05757761001586914 length of segment : 436 time for calcul the mask position with numpy : 0.0013456344604492188 nb_pixel_total : 40686 time to create 1 rle with old method : 0.044891357421875 length of segment : 272 time for calcul the mask position with numpy : 0.0004138946533203125 nb_pixel_total : 7646 time to create 1 rle with old method : 0.009086370468139648 length of segment : 90 time for calcul the mask position with numpy : 0.0038673877716064453 nb_pixel_total : 112092 time to create 1 rle with old method : 0.13136720657348633 length of segment : 432 time for calcul the mask position with numpy : 0.002382040023803711 nb_pixel_total : 37417 time to create 1 rle with old method : 0.0444033145904541 length of segment : 379 time for calcul the mask position with numpy : 0.0075986385345458984 nb_pixel_total : 152835 time to create 1 rle with new method : 0.012414693832397461 length of segment : 639 time for calcul the mask position with numpy : 0.0007905960083007812 nb_pixel_total : 23945 time to create 1 rle with old method : 0.02793264389038086 length of segment : 230 time for calcul the mask position with numpy : 0.003307342529296875 nb_pixel_total : 90579 time to create 1 rle with old method : 0.1067819595336914 length of segment : 712 time for calcul the mask position with numpy : 0.0006055831909179688 nb_pixel_total : 11725 time to create 1 rle with old method : 0.013213396072387695 length of segment : 129 time for calcul the mask position with numpy : 0.0003495216369628906 nb_pixel_total : 7270 time to create 1 rle with old method : 0.008890867233276367 length of segment : 95 time for calcul the mask position with numpy : 0.0011289119720458984 nb_pixel_total : 35068 time to create 1 rle with old method : 0.03875899314880371 length of segment : 305 time for calcul the mask position with numpy : 0.0003409385681152344 nb_pixel_total : 15978 time to create 1 rle with old method : 0.018186569213867188 length of segment : 164 time for calcul the mask position with numpy : 0.0017404556274414062 nb_pixel_total : 45522 time to create 1 rle with old method : 0.05166506767272949 length of segment : 476 time for calcul the mask position with numpy : 0.0010058879852294922 nb_pixel_total : 15850 time to create 1 rle with old method : 0.017908811569213867 length of segment : 256 time for calcul the mask position with numpy : 0.0007653236389160156 nb_pixel_total : 17473 time to create 1 rle with old method : 0.020334482192993164 length of segment : 190 time for calcul the mask position with numpy : 0.0018155574798583984 nb_pixel_total : 56222 time to create 1 rle with old method : 0.06404805183410645 length of segment : 380 time for calcul the mask position with numpy : 0.0002994537353515625 nb_pixel_total : 9433 time to create 1 rle with old method : 0.011393308639526367 length of segment : 89 time for calcul the mask position with numpy : 0.00039386749267578125 nb_pixel_total : 12372 time to create 1 rle with old method : 0.013703584671020508 length of segment : 214 time for calcul the mask position with numpy : 0.001871347427368164 nb_pixel_total : 65910 time to create 1 rle with old method : 0.08553004264831543 length of segment : 291 time for calcul the mask position with numpy : 0.001008749008178711 nb_pixel_total : 42436 time to create 1 rle with old method : 0.05341029167175293 length of segment : 323 time for calcul the mask position with numpy : 0.0054929256439208984 nb_pixel_total : 118546 time to create 1 rle with old method : 0.14738249778747559 length of segment : 805 time for calcul the mask position with numpy : 0.002346515655517578 nb_pixel_total : 62082 time to create 1 rle with old method : 0.07603240013122559 length of segment : 365 time for calcul the mask position with numpy : 0.00017881393432617188 nb_pixel_total : 6076 time to create 1 rle with old method : 0.007240772247314453 length of segment : 179 time for calcul the mask position with numpy : 0.0014257431030273438 nb_pixel_total : 45229 time to create 1 rle with old method : 0.0537264347076416 length of segment : 270 time for calcul the mask position with numpy : 0.0011589527130126953 nb_pixel_total : 28003 time to create 1 rle with old method : 0.03253817558288574 length of segment : 242 time for calcul the mask position with numpy : 0.0003554821014404297 nb_pixel_total : 5539 time to create 1 rle with old method : 0.0069122314453125 length of segment : 116 time for calcul the mask position with numpy : 0.001760244369506836 nb_pixel_total : 35629 time to create 1 rle with old method : 0.0419003963470459 length of segment : 268 time for calcul the mask position with numpy : 0.0009946823120117188 nb_pixel_total : 17671 time to create 1 rle with old method : 0.02882242202758789 length of segment : 198 time for calcul the mask position with numpy : 0.004272937774658203 nb_pixel_total : 123894 time to create 1 rle with old method : 0.15208983421325684 length of segment : 415 time for calcul the mask position with numpy : 0.0007872581481933594 nb_pixel_total : 13925 time to create 1 rle with old method : 0.023635387420654297 length of segment : 132 time for calcul the mask position with numpy : 0.005750417709350586 nb_pixel_total : 174561 time to create 1 rle with new method : 0.009198427200317383 length of segment : 384 time for calcul the mask position with numpy : 0.0006306171417236328 nb_pixel_total : 12749 time to create 1 rle with old method : 0.015271663665771484 length of segment : 189 time for calcul the mask position with numpy : 0.0008754730224609375 nb_pixel_total : 14739 time to create 1 rle with old method : 0.017764568328857422 length of segment : 219 time for calcul the mask position with numpy : 0.001710653305053711 nb_pixel_total : 47317 time to create 1 rle with old method : 0.05694937705993652 length of segment : 302 time for calcul the mask position with numpy : 0.0022125244140625 nb_pixel_total : 47660 time to create 1 rle with old method : 0.06060981750488281 length of segment : 378 time for calcul the mask position with numpy : 0.001949310302734375 nb_pixel_total : 35079 time to create 1 rle with old method : 0.04090619087219238 length of segment : 310 time for calcul the mask position with numpy : 0.001814126968383789 nb_pixel_total : 37909 time to create 1 rle with old method : 0.0492854118347168 length of segment : 340 time for calcul the mask position with numpy : 0.0005540847778320312 nb_pixel_total : 17399 time to create 1 rle with old method : 0.021556615829467773 length of segment : 142 time for calcul the mask position with numpy : 0.0009632110595703125 nb_pixel_total : 28196 time to create 1 rle with old method : 0.0363314151763916 length of segment : 200 time for calcul the mask position with numpy : 0.0020575523376464844 nb_pixel_total : 59350 time to create 1 rle with old method : 0.06998848915100098 length of segment : 347 time for calcul the mask position with numpy : 0.0006856918334960938 nb_pixel_total : 14519 time to create 1 rle with old method : 0.018896818161010742 length of segment : 184 time for calcul the mask position with numpy : 0.0015969276428222656 nb_pixel_total : 39395 time to create 1 rle with old method : 0.048476457595825195 length of segment : 219 time for calcul the mask position with numpy : 0.0011548995971679688 nb_pixel_total : 19468 time to create 1 rle with old method : 0.02849102020263672 length of segment : 149 time spent for convertir_results : 12.327975273132324 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 116 chid ids of type : 3594 Number RLEs to save : 31051 save missing photos in datou_result : time spend for datou_step_exec : 74.36413741111755 time spend to save output : 11.073376417160034 total time spend for step 1 : 85.43751382827759 step2:crop_condition Thu Apr 10 16:11:56 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 7 ! batch 1 Loaded 116 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 97 About to insert : list_path_to_insert length 97 new photo from crops ! About to upload 97 photos upload in portfolio : 3736932 init cache_photo without model_param we have 97 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1744294342_3182161 we have uploaded 97 photos in the portfolio 3736932 time of upload the photos Elapsed time : 45.77229118347168 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 15 About to insert : list_path_to_insert length 15 new photo from crops ! About to upload 15 photos upload in portfolio : 3736932 init cache_photo without model_param we have 15 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1744294392_3182161 we have uploaded 15 photos in the portfolio 3736932 time of upload the photos Elapsed time : 4.185281276702881 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1744294398_3182161 we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.5920367240905762 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 3736932 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1744294401_3182161 we have uploaded 3 photos in the portfolio 3736932 time of upload the photos Elapsed time : 3.7851014137268066 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1351168258, 1351168255, 1351168240, 1351168237, 1351168234, 1351168220, 1351168204] Looping around the photos to save general results len do output : 116 /1351192445Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192448Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192449Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192450Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192453Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192454Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192455Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192458Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192460Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192461Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192464Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192465Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192468Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192469Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192472Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192475Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192477Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192481Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192482Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192483Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192486Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192487Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192488Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192491Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192492Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192493Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192496Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192497Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192498Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192501Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192502Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192503Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192506Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192507Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192509Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192511Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192513Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192514Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192516Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192520Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192522Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192524Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192528Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192531Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192534Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192539Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192541Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192543Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192545Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192546Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192549Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192551Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192552Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192555Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192556Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192557Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192560Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192561Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192562Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192565Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192566Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192567Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192570Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192571Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192572Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192574Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192578Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192582Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192591Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192594Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192598Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192599Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192600Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192601Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192602Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192603Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192604Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192606Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192610Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192612Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192613Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192614Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192615Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192616Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192617Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192618Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192619Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192620Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192621Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192622Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192623Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192624Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192625Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192626Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192627Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192628Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192629Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192690Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192691Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192692Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192693Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192694Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192695Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192696Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192697Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192698Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192699Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192700Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192701Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192702Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192703Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192704Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192715Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192731Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192732Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351192733Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168258', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168255', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168240', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168237', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168234', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168220', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168204', None, None, None, None, None, '2740212') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 355 time used for this insertion : 0.031117677688598633 save_final save missing photos in datou_result : time spend for datou_step_exec : 87.8561692237854 time spend to save output : 0.03571152687072754 total time spend for step 2 : 87.89188075065613 step3:rle_unique_nms_with_priority Thu Apr 10 16:13:24 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 116 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 9 nb_hashtags : 4 time to prepare the origin masks : 7.535594701766968 time for calcul the mask position with numpy : 0.9145610332489014 nb_pixel_total : 7705503 time to create 1 rle with new method : 1.092325210571289 time for calcul the mask position with numpy : 0.04240536689758301 nb_pixel_total : 14062 time to create 1 rle with old method : 0.017089128494262695 time for calcul the mask position with numpy : 0.04304671287536621 nb_pixel_total : 109151 time to create 1 rle with old method : 0.13276147842407227 time for calcul the mask position with numpy : 0.04669332504272461 nb_pixel_total : 90332 time to create 1 rle with old method : 0.12151980400085449 time for calcul the mask position with numpy : 0.03414630889892578 nb_pixel_total : 15644 time to create 1 rle with old method : 0.01836419105529785 time for calcul the mask position with numpy : 0.027232646942138672 nb_pixel_total : 156479 time to create 1 rle with new method : 0.8331863880157471 time for calcul the mask position with numpy : 0.02929520606994629 nb_pixel_total : 20321 time to create 1 rle with old method : 0.02460455894470215 time for calcul the mask position with numpy : 0.027688026428222656 nb_pixel_total : 25436 time to create 1 rle with old method : 0.029676198959350586 time for calcul the mask position with numpy : 0.029065370559692383 nb_pixel_total : 78934 time to create 1 rle with old method : 0.09416413307189941 time for calcul the mask position with numpy : 0.02686309814453125 nb_pixel_total : 78538 time to create 1 rle with old method : 0.09255623817443848 create new chi : 3.7472000122070312 time to delete rle : 0.025684356689453125 batch 1 Loaded 19 chid ids of type : 3594 ++++++++++++Number RLEs to save : 7886 TO DO : save crop sub photo not yet done ! save time : 4.951005458831787 nb_obj : 19 nb_hashtags : 2 time to prepare the origin masks : 9.768486261367798 time for calcul the mask position with numpy : 0.5636279582977295 nb_pixel_total : 7387051 time to create 1 rle with new method : 0.8287336826324463 time for calcul the mask position with numpy : 0.03997087478637695 nb_pixel_total : 7291 time to create 1 rle with old method : 0.012408256530761719 time for calcul the mask position with numpy : 0.02497410774230957 nb_pixel_total : 26438 time to create 1 rle with old method : 0.030277013778686523 time for calcul the mask position with numpy : 0.023854494094848633 nb_pixel_total : 14512 time to create 1 rle with old method : 0.01687479019165039 time for calcul the mask position with numpy : 0.025553226470947266 nb_pixel_total : 15622 time to create 1 rle with old method : 0.01801300048828125 time for calcul the mask position with numpy : 0.025668621063232422 nb_pixel_total : 37384 time to create 1 rle with old method : 0.04261136054992676 time for calcul the mask position with numpy : 0.024839401245117188 nb_pixel_total : 23379 time to create 1 rle with old method : 0.026617765426635742 time for calcul the mask position with numpy : 0.026439905166625977 nb_pixel_total : 80484 time to create 1 rle with old method : 0.09388017654418945 time for calcul the mask position with numpy : 0.025275707244873047 nb_pixel_total : 44576 time to create 1 rle with old method : 0.05177497863769531 time for calcul the mask position with numpy : 0.024660587310791016 nb_pixel_total : 15068 time to create 1 rle with old method : 0.017429113388061523 time for calcul the mask position with numpy : 0.02561664581298828 nb_pixel_total : 11193 time to create 1 rle with old method : 0.0126800537109375 time for calcul the mask position with numpy : 0.024159669876098633 nb_pixel_total : 3953 time to create 1 rle with old method : 0.004596233367919922 time for calcul the mask position with numpy : 0.02407240867614746 nb_pixel_total : 36757 time to create 1 rle with old method : 0.042104482650756836 time for calcul the mask position with numpy : 0.024789094924926758 nb_pixel_total : 131687 time to create 1 rle with old method : 0.15118646621704102 time for calcul the mask position with numpy : 0.02650928497314453 nb_pixel_total : 17827 time to create 1 rle with old method : 0.02089977264404297 time for calcul the mask position with numpy : 0.027204036712646484 nb_pixel_total : 42592 time to create 1 rle with old method : 0.05008578300476074 time for calcul the mask position with numpy : 0.027034997940063477 nb_pixel_total : 113194 time to create 1 rle with old method : 0.13049840927124023 time for calcul the mask position with numpy : 0.02555060386657715 nb_pixel_total : 44517 time to create 1 rle with old method : 0.0521702766418457 time for calcul the mask position with numpy : 0.026671171188354492 nb_pixel_total : 28927 time to create 1 rle with old method : 0.033685922622680664 time for calcul the mask position with numpy : 0.027765750885009766 nb_pixel_total : 211948 time to create 1 rle with new method : 0.8734996318817139 create new chi : 3.6666104793548584 time to delete rle : 0.0024030208587646484 batch 1 Loaded 39 chid ids of type : 3594 +++++++++++++++++++Number RLEs to save : 11304 TO DO : save crop sub photo not yet done ! save time : 2.482713460922241 nb_obj : 21 nb_hashtags : 2 time to prepare the origin masks : 12.408339500427246 time for calcul the mask position with numpy : 1.117682695388794 nb_pixel_total : 7647919 time to create 1 rle with new method : 0.9449241161346436 time for calcul the mask position with numpy : 0.043973445892333984 nb_pixel_total : 8473 time to create 1 rle with old method : 0.009907722473144531 time for calcul the mask position with numpy : 0.04722952842712402 nb_pixel_total : 34497 time to create 1 rle with old method : 0.040367841720581055 time for calcul the mask position with numpy : 0.04437661170959473 nb_pixel_total : 10770 time to create 1 rle with old method : 0.01276254653930664 time for calcul the mask position with numpy : 0.04238176345825195 nb_pixel_total : 11260 time to create 1 rle with old method : 0.018956422805786133 time for calcul the mask position with numpy : 0.045857906341552734 nb_pixel_total : 31993 time to create 1 rle with old method : 0.03926229476928711 time for calcul the mask position with numpy : 0.040787696838378906 nb_pixel_total : 69163 time to create 1 rle with old method : 0.07950067520141602 time for calcul the mask position with numpy : 0.04218864440917969 nb_pixel_total : 92686 time to create 1 rle with old method : 0.10802888870239258 time for calcul the mask position with numpy : 0.04212546348571777 nb_pixel_total : 22120 time to create 1 rle with old method : 0.026489973068237305 time for calcul the mask position with numpy : 0.039998531341552734 nb_pixel_total : 75949 time to create 1 rle with old method : 0.08860993385314941 time for calcul the mask position with numpy : 0.040454864501953125 nb_pixel_total : 31878 time to create 1 rle with old method : 0.03721284866333008 time for calcul the mask position with numpy : 0.028764009475708008 nb_pixel_total : 6684 time to create 1 rle with old method : 0.0076711177825927734 time for calcul the mask position with numpy : 0.027266502380371094 nb_pixel_total : 22615 time to create 1 rle with old method : 0.026429176330566406 time for calcul the mask position with numpy : 0.026152849197387695 nb_pixel_total : 28935 time to create 1 rle with old method : 0.03491377830505371 time for calcul the mask position with numpy : 0.026770591735839844 nb_pixel_total : 10905 time to create 1 rle with old method : 0.013244867324829102 time for calcul the mask position with numpy : 0.028641462326049805 nb_pixel_total : 18118 time to create 1 rle with old method : 0.021583080291748047 time for calcul the mask position with numpy : 0.03119635581970215 nb_pixel_total : 11493 time to create 1 rle with old method : 0.013944149017333984 time for calcul the mask position with numpy : 0.03366971015930176 nb_pixel_total : 12693 time to create 1 rle with old method : 0.015903949737548828 time for calcul the mask position with numpy : 0.02819657325744629 nb_pixel_total : 53813 time to create 1 rle with old method : 0.06408905982971191 time for calcul the mask position with numpy : 0.03879404067993164 nb_pixel_total : 19591 time to create 1 rle with old method : 0.0227963924407959 time for calcul the mask position with numpy : 0.0492551326751709 nb_pixel_total : 42867 time to create 1 rle with old method : 0.05561208724975586 time for calcul the mask position with numpy : 0.054982662200927734 nb_pixel_total : 29978 time to create 1 rle with old method : 0.046254873275756836 create new chi : 3.70324969291687 time to delete rle : 0.002591848373413086 batch 1 Loaded 43 chid ids of type : 3594 ++++++++++++++++++++++++++++Number RLEs to save : 12135 TO DO : save crop sub photo not yet done ! save time : 4.831880807876587 nb_obj : 20 nb_hashtags : 3 time to prepare the origin masks : 11.280662059783936 time for calcul the mask position with numpy : 1.0590176582336426 nb_pixel_total : 7283670 time to create 1 rle with new method : 1.1391220092773438 time for calcul the mask position with numpy : 0.03979229927062988 nb_pixel_total : 6750 time to create 1 rle with old method : 0.0076215267181396484 time for calcul the mask position with numpy : 0.04104185104370117 nb_pixel_total : 46087 time to create 1 rle with old method : 0.05178046226501465 time for calcul the mask position with numpy : 0.04143261909484863 nb_pixel_total : 3009 time to create 1 rle with old method : 0.003571033477783203 time for calcul the mask position with numpy : 0.04053616523742676 nb_pixel_total : 5628 time to create 1 rle with old method : 0.006379604339599609 time for calcul the mask position with numpy : 0.039099693298339844 nb_pixel_total : 13110 time to create 1 rle with old method : 0.014789104461669922 time for calcul the mask position with numpy : 0.038278818130493164 nb_pixel_total : 541 time to create 1 rle with old method : 0.0007348060607910156 time for calcul the mask position with numpy : 0.03952383995056152 nb_pixel_total : 38514 time to create 1 rle with old method : 0.04353523254394531 time for calcul the mask position with numpy : 0.03957104682922363 nb_pixel_total : 69084 time to create 1 rle with old method : 0.07595252990722656 time for calcul the mask position with numpy : 0.039546966552734375 nb_pixel_total : 13988 time to create 1 rle with old method : 0.016126155853271484 time for calcul the mask position with numpy : 0.03863358497619629 nb_pixel_total : 23802 time to create 1 rle with old method : 0.026827096939086914 time for calcul the mask position with numpy : 0.026479721069335938 nb_pixel_total : 15461 time to create 1 rle with old method : 0.017409086227416992 time for calcul the mask position with numpy : 0.027037382125854492 nb_pixel_total : 335071 time to create 1 rle with new method : 0.7248013019561768 time for calcul the mask position with numpy : 0.029488801956176758 nb_pixel_total : 73385 time to create 1 rle with old method : 0.08789682388305664 time for calcul the mask position with numpy : 0.02776789665222168 nb_pixel_total : 47832 time to create 1 rle with old method : 0.05726027488708496 time for calcul the mask position with numpy : 0.03083038330078125 nb_pixel_total : 198358 time to create 1 rle with new method : 0.641028881072998 time for calcul the mask position with numpy : 0.03142857551574707 nb_pixel_total : 64735 time to create 1 rle with old method : 0.07935500144958496 time for calcul the mask position with numpy : 0.028102397918701172 nb_pixel_total : 5307 time to create 1 rle with old method : 0.0063326358795166016 time for calcul the mask position with numpy : 0.0298464298248291 nb_pixel_total : 9893 time to create 1 rle with old method : 0.011682271957397461 time for calcul the mask position with numpy : 0.028255462646484375 nb_pixel_total : 20127 time to create 1 rle with old method : 0.023790836334228516 time for calcul the mask position with numpy : 0.027704954147338867 nb_pixel_total : 20048 time to create 1 rle with old method : 0.028096675872802734 create new chi : 4.917391300201416 time to delete rle : 0.0021507740020751953 batch 1 Loaded 41 chid ids of type : 3594 +++++++++++++++++++++++++++++Number RLEs to save : 12317 TO DO : save crop sub photo not yet done ! save time : 6.545608997344971 nb_obj : 19 nb_hashtags : 2 time to prepare the origin masks : 10.478240251541138 time for calcul the mask position with numpy : 0.8479290008544922 nb_pixel_total : 7480246 time to create 1 rle with new method : 1.050729751586914 time for calcul the mask position with numpy : 0.02723407745361328 nb_pixel_total : 17473 time to create 1 rle with old method : 0.021642208099365234 time for calcul the mask position with numpy : 0.026715993881225586 nb_pixel_total : 15850 time to create 1 rle with old method : 0.01987457275390625 time for calcul the mask position with numpy : 0.031034469604492188 nb_pixel_total : 45522 time to create 1 rle with old method : 0.05683326721191406 time for calcul the mask position with numpy : 0.030277252197265625 nb_pixel_total : 15978 time to create 1 rle with old method : 0.03453850746154785 time for calcul the mask position with numpy : 0.03985714912414551 nb_pixel_total : 35068 time to create 1 rle with old method : 0.03974771499633789 time for calcul the mask position with numpy : 0.043910980224609375 nb_pixel_total : 7270 time to create 1 rle with old method : 0.008435249328613281 time for calcul the mask position with numpy : 0.04749155044555664 nb_pixel_total : 11725 time to create 1 rle with old method : 0.01885700225830078 time for calcul the mask position with numpy : 0.04354381561279297 nb_pixel_total : 71214 time to create 1 rle with old method : 0.14262938499450684 time for calcul the mask position with numpy : 0.05106806755065918 nb_pixel_total : 23768 time to create 1 rle with old method : 0.027535200119018555 time for calcul the mask position with numpy : 0.04212307929992676 nb_pixel_total : 152835 time to create 1 rle with new method : 0.6984133720397949 time for calcul the mask position with numpy : 0.044425249099731445 nb_pixel_total : 37417 time to create 1 rle with old method : 0.05295681953430176 time for calcul the mask position with numpy : 0.07447052001953125 nb_pixel_total : 112092 time to create 1 rle with old method : 0.13061857223510742 time for calcul the mask position with numpy : 0.04171109199523926 nb_pixel_total : 7646 time to create 1 rle with old method : 0.009165525436401367 time for calcul the mask position with numpy : 0.04183077812194824 nb_pixel_total : 40686 time to create 1 rle with old method : 0.04701375961303711 time for calcul the mask position with numpy : 0.04156088829040527 nb_pixel_total : 53050 time to create 1 rle with old method : 0.06168007850646973 time for calcul the mask position with numpy : 0.04253840446472168 nb_pixel_total : 27695 time to create 1 rle with old method : 0.03437089920043945 time for calcul the mask position with numpy : 0.04190206527709961 nb_pixel_total : 15725 time to create 1 rle with old method : 0.018459558486938477 time for calcul the mask position with numpy : 0.04546356201171875 nb_pixel_total : 12499 time to create 1 rle with old method : 0.015537738800048828 time for calcul the mask position with numpy : 0.04499506950378418 nb_pixel_total : 110641 time to create 1 rle with old method : 0.12888407707214355 create new chi : 4.348286390304565 time to delete rle : 0.0031778812408447266 batch 1 Loaded 39 chid ids of type : 3594 ++++++++++++++++++++++++++++++Number RLEs to save : 13340 TO DO : save crop sub photo not yet done ! save time : 7.236349821090698 nb_obj : 11 nb_hashtags : 1 time to prepare the origin masks : 5.882615804672241 time for calcul the mask position with numpy : 1.0086441040039062 nb_pixel_total : 7847880 time to create 1 rle with new method : 0.9104959964752197 time for calcul the mask position with numpy : 0.04046773910522461 nb_pixel_total : 5539 time to create 1 rle with old method : 0.006316184997558594 time for calcul the mask position with numpy : 0.04012036323547363 nb_pixel_total : 28003 time to create 1 rle with old method : 0.03147387504577637 time for calcul the mask position with numpy : 0.04025530815124512 nb_pixel_total : 45116 time to create 1 rle with old method : 0.050731658935546875 time for calcul the mask position with numpy : 0.036832571029663086 nb_pixel_total : 1331 time to create 1 rle with old method : 0.0015308856964111328 time for calcul the mask position with numpy : 0.03783607482910156 nb_pixel_total : 62082 time to create 1 rle with old method : 0.0700845718383789 time for calcul the mask position with numpy : 0.035843849182128906 nb_pixel_total : 118546 time to create 1 rle with old method : 0.16328215599060059 time for calcul the mask position with numpy : 0.02736639976501465 nb_pixel_total : 41966 time to create 1 rle with old method : 0.04889249801635742 time for calcul the mask position with numpy : 0.02664661407470703 nb_pixel_total : 65910 time to create 1 rle with old method : 0.0757453441619873 time for calcul the mask position with numpy : 0.027390480041503906 nb_pixel_total : 12372 time to create 1 rle with old method : 0.01472616195678711 time for calcul the mask position with numpy : 0.02462935447692871 nb_pixel_total : 9433 time to create 1 rle with old method : 0.010929107666015625 time for calcul the mask position with numpy : 0.024554729461669922 nb_pixel_total : 56222 time to create 1 rle with old method : 0.06579422950744629 create new chi : 2.8689277172088623 time to delete rle : 0.0019860267639160156 batch 1 Loaded 23 chid ids of type : 3594 +++++++++++++++++Number RLEs to save : 8415 TO DO : save crop sub photo not yet done ! save time : 4.207814693450928 nb_obj : 17 nb_hashtags : 2 time to prepare the origin masks : 10.90405559539795 time for calcul the mask position with numpy : 0.4328737258911133 nb_pixel_total : 7554940 time to create 1 rle with new method : 0.8876707553863525 time for calcul the mask position with numpy : 0.041944265365600586 nb_pixel_total : 19468 time to create 1 rle with old method : 0.02350139617919922 time for calcul the mask position with numpy : 0.04805469512939453 nb_pixel_total : 39395 time to create 1 rle with old method : 0.050278425216674805 time for calcul the mask position with numpy : 0.04818582534790039 nb_pixel_total : 14519 time to create 1 rle with old method : 0.018792152404785156 time for calcul the mask position with numpy : 0.04164934158325195 nb_pixel_total : 59350 time to create 1 rle with old method : 0.06779313087463379 time for calcul the mask position with numpy : 0.03977680206298828 nb_pixel_total : 28196 time to create 1 rle with old method : 0.03274965286254883 time for calcul the mask position with numpy : 0.04024362564086914 nb_pixel_total : 17399 time to create 1 rle with old method : 0.019387006759643555 time for calcul the mask position with numpy : 0.038979291915893555 nb_pixel_total : 37909 time to create 1 rle with old method : 0.043280601501464844 time for calcul the mask position with numpy : 0.02456355094909668 nb_pixel_total : 35079 time to create 1 rle with old method : 0.03930830955505371 time for calcul the mask position with numpy : 0.02538466453552246 nb_pixel_total : 47660 time to create 1 rle with old method : 0.054157257080078125 time for calcul the mask position with numpy : 0.025050640106201172 nb_pixel_total : 47317 time to create 1 rle with old method : 0.0531768798828125 time for calcul the mask position with numpy : 0.024512052536010742 nb_pixel_total : 14739 time to create 1 rle with old method : 0.016704559326171875 time for calcul the mask position with numpy : 0.02760481834411621 nb_pixel_total : 12749 time to create 1 rle with old method : 0.01446533203125 time for calcul the mask position with numpy : 0.02619338035583496 nb_pixel_total : 174561 time to create 1 rle with new method : 0.6733086109161377 time for calcul the mask position with numpy : 0.02417778968811035 nb_pixel_total : 13925 time to create 1 rle with old method : 0.015566587448120117 time for calcul the mask position with numpy : 0.024413108825683594 nb_pixel_total : 123894 time to create 1 rle with old method : 0.13905787467956543 time for calcul the mask position with numpy : 0.02539539337158203 nb_pixel_total : 17671 time to create 1 rle with old method : 0.02035045623779297 time for calcul the mask position with numpy : 0.02539658546447754 nb_pixel_total : 35629 time to create 1 rle with old method : 0.04081559181213379 create new chi : 3.275944471359253 time to delete rle : 0.0015778541564941406 batch 1 Loaded 35 chid ids of type : 3594 +++++++++++++++++++Number RLEs to save : 10912 TO DO : save crop sub photo not yet done ! save time : 7.089014530181885 map_output_result : {1351168258: (0.0, 'Should be the crop_list due to order', 0), 1351168255: (0.0, 'Should be the crop_list due to order', 0), 1351168240: (0.0, 'Should be the crop_list due to order', 0), 1351168237: (0.0, 'Should be the crop_list due to order', 0), 1351168234: (0.0, 'Should be the crop_list due to order', 0), 1351168220: (0.0, 'Should be the crop_list due to order', 0), 1351168204: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1351168258, 1351168255, 1351168240, 1351168237, 1351168234, 1351168220, 1351168204] Looping around the photos to save general results len do output : 7 /1351168258.Didn't retrieve data . /1351168255.Didn't retrieve data . /1351168240.Didn't retrieve data . /1351168237.Didn't retrieve data . /1351168234.Didn't retrieve data . /1351168220.Didn't retrieve data . /1351168204.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168258', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168255', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168240', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168237', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168234', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168220', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168204', None, None, None, None, None, '2740212') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 21 time used for this insertion : 0.015531778335571289 save_final save missing photos in datou_result : time spend for datou_step_exec : 133.4773941040039 time spend to save output : 0.01600050926208496 total time spend for step 3 : 133.493394613266 step4:ventilate_hashtags_in_portfolio Thu Apr 10 16:15:38 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 22230622 get user id for portfolio 22230622 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22230622 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('carton','pet_fonce','flou','autre','papier','background','mal_croppe','metal','pehd','pet_clair','environnement')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22230622 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('carton','pet_fonce','flou','autre','papier','background','mal_croppe','metal','pehd','pet_clair','environnement')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22230622 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('carton','pet_fonce','flou','autre','papier','background','mal_croppe','metal','pehd','pet_clair','environnement')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/22231443,22231444,22231445,22231446,22231447,22231448,22231449,22231450,22231451,22231452,22231453?tags=carton,pet_fonce,flou,autre,papier,background,mal_croppe,metal,pehd,pet_clair,environnement Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1351168258, 1351168255, 1351168240, 1351168237, 1351168234, 1351168220, 1351168204] Looping around the photos to save general results len do output : 1 /22230622. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168258', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168255', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168240', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168237', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168234', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168220', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168204', None, None, None, None, None, '2740212') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 8 time used for this insertion : 0.013249874114990234 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.8571491241455078 time spend to save output : 0.013573169708251953 total time spend for step 4 : 1.8707222938537598 step5:final Thu Apr 10 16:15:40 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1351168258: ('0.0887619702105379',), 1351168255: ('0.0887619702105379',), 1351168240: ('0.0887619702105379',), 1351168237: ('0.0887619702105379',), 1351168234: ('0.0887619702105379',), 1351168220: ('0.0887619702105379',), 1351168204: ('0.0887619702105379',)} new output for save of step final : {1351168258: ('0.0887619702105379',), 1351168255: ('0.0887619702105379',), 1351168240: ('0.0887619702105379',), 1351168237: ('0.0887619702105379',), 1351168234: ('0.0887619702105379',), 1351168220: ('0.0887619702105379',), 1351168204: ('0.0887619702105379',)} [1351168258, 1351168255, 1351168240, 1351168237, 1351168234, 1351168220, 1351168204] Looping around the photos to save general results len do output : 7 /1351168258.Didn't retrieve data . /1351168255.Didn't retrieve data . /1351168240.Didn't retrieve data . /1351168237.Didn't retrieve data . /1351168234.Didn't retrieve data . /1351168220.Didn't retrieve data . /1351168204.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168258', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168255', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168240', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168237', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168234', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168220', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168204', None, None, None, None, None, '2740212') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 21 time used for this insertion : 0.014900445938110352 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.1213228702545166 time spend to save output : 0.017210960388183594 total time spend for step 5 : 0.1385338306427002 step6:blur_detection Thu Apr 10 16:15:40 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1744294230_3182161_1351168258_67172d68c1396078242eb73810ffbb30.jpg resize: (2160, 3840) 1351168258 -5.234168114201895 treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c.jpg resize: (2160, 3840) 1351168255 -5.978024745462436 treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5.jpg resize: (2160, 3840) 1351168240 -6.043618625917765 treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b.jpg resize: (2160, 3840) 1351168237 -6.203897258699527 treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a.jpg resize: (2160, 3840) 1351168234 -6.158585330060653 treat image : temp/1744294230_3182161_1351168220_79793575f6d7dcdcbb76a4b544969781.jpg resize: (2160, 3840) 1351168220 -6.127721359023935 treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d.jpg resize: (2160, 3840) 1351168204 -6.065713419135692 treat image : temp/1744294230_3182161_1351168258_67172d68c1396078242eb73810ffbb30_rle_crop_3754218864_0.png resize: (265, 544) 1351192445 -4.530548736017812 treat image : temp/1744294230_3182161_1351168258_67172d68c1396078242eb73810ffbb30_rle_crop_3754218865_0.png resize: (362, 371) 1351192448 -3.0698270995232324 treat image : temp/1744294230_3182161_1351168258_67172d68c1396078242eb73810ffbb30_rle_crop_3754218868_0.png resize: (560, 419) 1351192449 -3.823932635837733 treat image : temp/1744294230_3182161_1351168258_67172d68c1396078242eb73810ffbb30_rle_crop_3754218871_0.png resize: (307, 478) 1351192450 -4.5937966307267555 treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218873_0.png resize: (640, 629) 1351192453 -2.3076359792325785 treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218875_0.png resize: (202, 259) 1351192454 -3.061018720405569 treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218876_0.png resize: (393, 406) 1351192455 -2.475259652772758 treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218877_0.png resize: (221, 283) 1351192458 -3.9684670018402866 treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218878_0.png resize: (151, 244) 1351192460 -3.8133984778556616 treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218879_0.png resize: (437, 480) 1351192461 -1.3703713474360528 treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218880_0.png resize: (237, 241) 1351192464 -1.6220614341968926 treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218881_0.png resize: (45, 113) 1351192465 1.2997088249370936 treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218883_0.png resize: (120, 175) 1351192468 -4.038821588975071 treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218884_0.png resize: (265, 238) 1351192469 -3.2063847797209855 treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218885_0.png resize: (322, 362) 1351192472 -2.627542735267804 treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218886_0.png resize: (163, 216) 1351192475 -2.9273910359687103 treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218887_0.png resize: (185, 272) 1351192477 -3.496750518463021 treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218888_0.png resize: (204, 106) 1351192481 -3.294320129113731 treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218890_0.png resize: (183, 220) 1351192482 -0.2638257694743422 treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218891_0.png resize: (127, 119) 1351192483 -1.9353179322514351 treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218893_0.png resize: (410, 149) 1351192486 -4.241985564109613 treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218894_0.png resize: (109, 221) 1351192487 -3.3582134819832032 treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218895_0.png resize: (207, 384) 1351192488 -2.1233103924314825 treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218896_0.png resize: (149, 132) 1351192491 -2.885022633441712 treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218897_0.png resize: (146, 129) 1351192492 -4.009845017385786 treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218898_0.png resize: (202, 105) 1351192493 -1.146674419413225 treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218899_0.png resize: (109, 133) 1351192496 -3.1836632833582965 treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218900_0.png resize: (263, 145) 1351192497 -3.698606246541867 treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218902_0.png resize: (148, 75) 1351192498 -2.8774770973338764 treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218903_0.png resize: (243, 192) 1351192501 -3.556375963215702 treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218905_0.png resize: (228, 148) 1351192502 -3.4675097287105103 treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218906_0.png resize: (573, 218) 1351192503 -4.465489270737738 treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218907_0.png resize: (340, 268) 1351192506 -3.9580076230343573 treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218908_0.png resize: (211, 218) 1351192507 -0.44423083097100213 treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218909_0.png resize: (99, 130) 1351192509 -3.2046161571168055 treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218910_0.png resize: (107, 139) 1351192511 -3.383120324502809 treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218911_0.png resize: (258, 290) 1351192513 -3.475492431572519 treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218912_0.png resize: (138, 92) 1351192514 -2.925372435451318 treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218913_0.png resize: (202, 129) 1351192516 -2.2399028294149224 treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218914_0.png resize: (201, 148) 1351192520 -3.5647695615118473 treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218915_0.png resize: (78, 175) 1351192522 -2.356596172553022 treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218916_0.png resize: (112, 72) 1351192524 -0.3010768376926556 treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218917_0.png resize: (357, 314) 1351192528 -2.1253591739949798 treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218918_0.png resize: (476, 608) 1351192531 -4.609293690076798 treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218919_0.png resize: (238, 258) 1351192534 -2.416942422123261 treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218920_0.png resize: (310, 304) 1351192539 -2.9819013312295404 treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218921_0.png resize: (931, 613) 1351192541 -2.4051910005667616 treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218922_0.png resize: (85, 244) 1351192543 -4.584382620560373 treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218924_0.png resize: (157, 113) 1351192545 -4.108372258924535 treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218925_0.png resize: (338, 293) 1351192546 -3.2389598722932833 treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218927_0.png resize: (92, 195) 1351192549 -0.8410880029647985 treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218928_0.png resize: (153, 148) 1351192551 -3.208062201689772 treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218929_0.png resize: (128, 58) 1351192552 -1.6451242201127976 treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218930_0.png resize: (96, 39) 1351192555 -3.2480341268807673 treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218932_0.png resize: (84, 104) 1351192556 -3.143894723271167 treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218933_0.png resize: (407, 421) 1351192557 -3.8649346209816713 treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218934_0.png resize: (117, 131) 1351192560 -1.2452320493684028 treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218935_0.png resize: (242, 154) 1351192561 -2.6139681169349007 treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218936_0.png resize: (200, 307) 1351192562 -1.0454755677638594 treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218937_0.png resize: (244, 325) 1351192565 -3.9925209506826844 treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218938_0.png resize: (268, 207) 1351192566 -4.158267988266169 treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218939_0.png resize: (87, 132) 1351192567 -4.373705431701869 treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218940_0.png resize: (385, 476) 1351192570 -3.407683516392981 treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218941_0.png resize: (260, 299) 1351192571 -4.058104473416078 treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218942_0.png resize: (577, 534) 1351192572 -3.757077158100539 treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218943_0.png resize: (198, 180) 1351192574 -3.860003490548732 treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218944_0.png resize: (437, 427) 1351192578 -2.9199473548072867 treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218945_0.png resize: (128, 165) 1351192582 -2.9142483631986096 treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218946_0.png resize: (95, 107) 1351192591 -1.5278765963303327 treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218947_0.png resize: (268, 209) 1351192594 -3.733810996106094 treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218948_0.png resize: (164, 138) 1351192598 -3.1034501152961638 treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218949_0.png resize: (344, 177) 1351192599 -4.4103309898943275 treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218950_0.png resize: (256, 81) 1351192600 -3.963275283086856 treat image : temp/1744294230_3182161_1351168220_79793575f6d7dcdcbb76a4b544969781_rle_crop_3754218952_0.png resize: (305, 297) 1351192601 -2.302548386581563 treat image : temp/1744294230_3182161_1351168220_79793575f6d7dcdcbb76a4b544969781_rle_crop_3754218953_0.png resize: (88, 143) 1351192602 -2.6200143808970293 treat image : temp/1744294230_3182161_1351168220_79793575f6d7dcdcbb76a4b544969781_rle_crop_3754218954_0.png resize: (180, 78) 1351192603 -0.567608903900898 treat image : temp/1744294230_3182161_1351168220_79793575f6d7dcdcbb76a4b544969781_rle_crop_3754218955_0.png resize: (286, 322) 1351192604 -1.6999192816352475 treat image : temp/1744294230_3182161_1351168220_79793575f6d7dcdcbb76a4b544969781_rle_crop_3754218956_0.png resize: (225, 264) 1351192606 -3.801748582221271 treat image : temp/1744294230_3182161_1351168220_79793575f6d7dcdcbb76a4b544969781_rle_crop_3754218957_0.png resize: (563, 474) 1351192610 -3.3742951613216907 treat image : temp/1744294230_3182161_1351168220_79793575f6d7dcdcbb76a4b544969781_rle_crop_3754218958_0.png resize: (352, 371) 1351192612 -2.413597745895161 treat image : temp/1744294230_3182161_1351168220_79793575f6d7dcdcbb76a4b544969781_rle_crop_3754218959_0.png resize: (154, 62) 1351192613 -1.7411593288876026 treat image : temp/1744294230_3182161_1351168220_79793575f6d7dcdcbb76a4b544969781_rle_crop_3754218960_0.png resize: (262, 254) 1351192614 -2.8904190256671 treat image : temp/1744294230_3182161_1351168220_79793575f6d7dcdcbb76a4b544969781_rle_crop_3754218961_0.png resize: (242, 187) 1351192615 -3.1089183951676174 treat image : temp/1744294230_3182161_1351168220_79793575f6d7dcdcbb76a4b544969781_rle_crop_3754218962_0.png resize: (115, 100) 1351192616 -3.717335771101392 treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218963_0.png resize: (268, 229) 1351192617 -2.507270574212025 treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218964_0.png resize: (195, 161) 1351192618 -3.6611025525943015 treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218965_0.png resize: (412, 381) 1351192619 -2.6532702593959447 treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218966_0.png resize: (130, 142) 1351192620 -2.5991721413207958 treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218968_0.png resize: (189, 77) 1351192621 -0.08945231922018862 treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218970_0.png resize: (297, 265) 1351192622 -3.3561243580366065 treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218972_0.png resize: (249, 212) 1351192623 -3.5590825955606196 treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218974_0.png resize: (139, 184) 1351192624 -1.5790524631080343 treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218975_0.png resize: (186, 184) 1351192625 -3.445940300115741 treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218976_0.png resize: (346, 214) 1351192626 -4.0067401210576215 treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218977_0.png resize: (170, 116) 1351192627 -3.5003569723376704 treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218978_0.png resize: (191, 288) 1351192628 -4.516540513416359 treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218979_0.png resize: (149, 196) 1351192629 -3.285104179429886 treat image : temp/1744294230_3182161_1351168258_67172d68c1396078242eb73810ffbb30_rle_crop_3754218866_0.png resize: (205, 190) 1351192690 -0.7348727297915733 treat image : temp/1744294230_3182161_1351168258_67172d68c1396078242eb73810ffbb30_rle_crop_3754218867_0.png resize: (141, 174) 1351192691 -4.153143861135299 treat image : temp/1744294230_3182161_1351168258_67172d68c1396078242eb73810ffbb30_rle_crop_3754218872_0.png resize: (92, 242) 1351192692 -3.331305464245906 treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218874_0.png resize: (198, 202) 1351192693 -3.7434231216440277 treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218882_0.png resize: (146, 109) 1351192694 -2.471409706665218 treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218889_0.png resize: (161, 113) 1351192695 -4.30565366211434 treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218892_0.png resize: (174, 250) 1351192696 -1.296111378728277 treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218901_0.png resize: (152, 185) 1351192697 -2.770754602886315 treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218904_0.png resize: (354, 325) 1351192698 -3.1811740341166916 treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218923_0.png resize: (203, 144) 1351192699 -2.2790268434811605 treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218951_0.png resize: (190, 132) 1351192700 -2.3223278216054983 treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218967_0.png resize: (369, 547) 1351192701 -4.450465453828039 treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218969_0.png resize: (219, 112) 1351192702 -2.9319006137576413 treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218971_0.png resize: (327, 241) 1351192703 -3.9163718716026885 treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218973_0.png resize: (332, 225) 1351192704 -2.936308718025388 treat image : temp/1744294230_3182161_1351168258_67172d68c1396078242eb73810ffbb30_rle_crop_3754218869_0.png resize: (141, 141) 1351192715 -4.181922684980132 treat image : temp/1744294230_3182161_1351168258_67172d68c1396078242eb73810ffbb30_rle_crop_3754218870_0.png resize: (531, 237) 1351192731 -2.9887053399719523 treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218926_0.png resize: (264, 197) 1351192732 -4.121200548249512 treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218931_0.png resize: (325, 225) 1351192733 -4.428613678248773 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 123 time used for this insertion : 0.4523656368255615 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 123 time used for this insertion : 0.032863616943359375 save missing photos in datou_result : time spend for datou_step_exec : 33.83944368362427 time spend to save output : 0.4903702735900879 total time spend for step 6 : 34.329813957214355 step7:brightness Thu Apr 10 16:16:14 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1744294230_3182161_1351168258_67172d68c1396078242eb73810ffbb30.jpg treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c.jpg treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5.jpg treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b.jpg treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a.jpg treat image : temp/1744294230_3182161_1351168220_79793575f6d7dcdcbb76a4b544969781.jpg treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d.jpg treat image : temp/1744294230_3182161_1351168258_67172d68c1396078242eb73810ffbb30_rle_crop_3754218864_0.png treat image : temp/1744294230_3182161_1351168258_67172d68c1396078242eb73810ffbb30_rle_crop_3754218865_0.png treat image : temp/1744294230_3182161_1351168258_67172d68c1396078242eb73810ffbb30_rle_crop_3754218868_0.png treat image : temp/1744294230_3182161_1351168258_67172d68c1396078242eb73810ffbb30_rle_crop_3754218871_0.png treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218873_0.png treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218875_0.png treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218876_0.png treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218877_0.png treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218878_0.png treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218879_0.png treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218880_0.png treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218881_0.png treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218883_0.png treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218884_0.png treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218885_0.png treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218886_0.png treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218887_0.png treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218888_0.png treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218890_0.png treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218891_0.png treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218893_0.png treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218894_0.png treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218895_0.png treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218896_0.png treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218897_0.png treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218898_0.png treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218899_0.png treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218900_0.png treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218902_0.png treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218903_0.png treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218905_0.png treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218906_0.png treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218907_0.png treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218908_0.png treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218909_0.png treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218910_0.png treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218911_0.png treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218912_0.png treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218913_0.png treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218914_0.png treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218915_0.png treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218916_0.png treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218917_0.png treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218918_0.png treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218919_0.png treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218920_0.png treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218921_0.png treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218922_0.png treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218924_0.png treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218925_0.png treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218927_0.png treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218928_0.png treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218929_0.png treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218930_0.png treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218932_0.png treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218933_0.png treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218934_0.png treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218935_0.png treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218936_0.png treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218937_0.png treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218938_0.png treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218939_0.png treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218940_0.png treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218941_0.png treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218942_0.png treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218943_0.png treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218944_0.png treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218945_0.png treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218946_0.png treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218947_0.png treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218948_0.png treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218949_0.png treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218950_0.png treat image : temp/1744294230_3182161_1351168220_79793575f6d7dcdcbb76a4b544969781_rle_crop_3754218952_0.png treat image : temp/1744294230_3182161_1351168220_79793575f6d7dcdcbb76a4b544969781_rle_crop_3754218953_0.png treat image : temp/1744294230_3182161_1351168220_79793575f6d7dcdcbb76a4b544969781_rle_crop_3754218954_0.png treat image : temp/1744294230_3182161_1351168220_79793575f6d7dcdcbb76a4b544969781_rle_crop_3754218955_0.png treat image : temp/1744294230_3182161_1351168220_79793575f6d7dcdcbb76a4b544969781_rle_crop_3754218956_0.png treat image : temp/1744294230_3182161_1351168220_79793575f6d7dcdcbb76a4b544969781_rle_crop_3754218957_0.png treat image : temp/1744294230_3182161_1351168220_79793575f6d7dcdcbb76a4b544969781_rle_crop_3754218958_0.png treat image : temp/1744294230_3182161_1351168220_79793575f6d7dcdcbb76a4b544969781_rle_crop_3754218959_0.png treat image : temp/1744294230_3182161_1351168220_79793575f6d7dcdcbb76a4b544969781_rle_crop_3754218960_0.png treat image : temp/1744294230_3182161_1351168220_79793575f6d7dcdcbb76a4b544969781_rle_crop_3754218961_0.png treat image : temp/1744294230_3182161_1351168220_79793575f6d7dcdcbb76a4b544969781_rle_crop_3754218962_0.png treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218963_0.png treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218964_0.png treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218965_0.png treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218966_0.png treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218968_0.png treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218970_0.png treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218972_0.png treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218974_0.png treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218975_0.png treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218976_0.png treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218977_0.png treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218978_0.png treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218979_0.png treat image : temp/1744294230_3182161_1351168258_67172d68c1396078242eb73810ffbb30_rle_crop_3754218866_0.png treat image : temp/1744294230_3182161_1351168258_67172d68c1396078242eb73810ffbb30_rle_crop_3754218867_0.png treat image : temp/1744294230_3182161_1351168258_67172d68c1396078242eb73810ffbb30_rle_crop_3754218872_0.png treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218874_0.png treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218882_0.png treat image : temp/1744294230_3182161_1351168255_d98bb6772c4bdaabd8de6e16feeabe9c_rle_crop_3754218889_0.png treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218892_0.png treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218901_0.png treat image : temp/1744294230_3182161_1351168240_1056ddb16af3777b8445bc52570f96f5_rle_crop_3754218904_0.png treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218923_0.png treat image : temp/1744294230_3182161_1351168234_9429f90a023e55650e9551bb3ecd7c4a_rle_crop_3754218951_0.png treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218967_0.png treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218969_0.png treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218971_0.png treat image : temp/1744294230_3182161_1351168204_0d56bc86f9f77618eff834dc34ea172d_rle_crop_3754218973_0.png treat image : temp/1744294230_3182161_1351168258_67172d68c1396078242eb73810ffbb30_rle_crop_3754218869_0.png treat image : temp/1744294230_3182161_1351168258_67172d68c1396078242eb73810ffbb30_rle_crop_3754218870_0.png treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218926_0.png treat image : temp/1744294230_3182161_1351168237_a8dee367ac4a9ca7bb8796d4644eef1b_rle_crop_3754218931_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 123 time used for this insertion : 0.08068370819091797 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 123 time used for this insertion : 2.567455291748047 save missing photos in datou_result : time spend for datou_step_exec : 7.643446683883667 time spend to save output : 2.6536684036254883 total time spend for step 7 : 10.297115087509155 step8:velours_tree Thu Apr 10 16:16:24 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.16287732124328613 time spend to save output : 9.202957153320312e-05 total time spend for step 8 : 0.16296935081481934 step9:send_mail_cod Thu Apr 10 16:16:24 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P22230622_10-04-2025_16_16_24.pdf 22231443 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette222314431744294584 22231444 imagette222314441744294586 22231445 imagette222314451744294586 22231446 imagette222314461744294586 22231447 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette222314471744294586 22231448 imagette222314481744294589 22231449 imagette222314491744294590 22231450 change filename to text .imagette222314501744294590 22231451 imagette222314511744294590 22231452 change filename to text .change filename to text .change filename to text .imagette222314521744294590 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=22230622 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/22231443,22231444,22231445,22231446,22231447,22231448,22231449,22231450,22231451,22231452,22231453?tags=carton,pet_fonce,flou,autre,papier,background,mal_croppe,metal,pehd,pet_clair,environnement args[1351168258] : ((1351168258, -5.234168114201895, 492609224), (1351168258, -0.5768572634188283, 501862349), '0.0887619702105379') We are sending mail with results at report@fotonower.com args[1351168255] : ((1351168255, -5.978024745462436, 492609224), (1351168255, 0.29166154945487116, 2107752395), '0.0887619702105379') We are sending mail with results at report@fotonower.com args[1351168240] : ((1351168240, -6.043618625917765, 492609224), (1351168240, 0.320589124627973, 2107752395), '0.0887619702105379') We are sending mail with results at report@fotonower.com args[1351168237] : ((1351168237, -6.203897258699527, 492609224), (1351168237, 0.24025204993468385, 2107752395), '0.0887619702105379') We are sending mail with results at report@fotonower.com args[1351168234] : ((1351168234, -6.158585330060653, 492609224), (1351168234, 0.13265100101829372, 2107752395), '0.0887619702105379') We are sending mail with results at report@fotonower.com args[1351168220] : ((1351168220, -6.127721359023935, 492609224), (1351168220, 0.2336708688489445, 2107752395), '0.0887619702105379') We are sending mail with results at report@fotonower.com args[1351168204] : ((1351168204, -6.065713419135692, 492609224), (1351168204, 0.18625334310967434, 2107752395), '0.0887619702105379') We are sending mail with results at report@fotonower.com refus_total : 0.0887619702105379 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=22230622 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22230622_10-04-2025_16_16_24.pdf results_Auto_P22230622_10-04-2025_16_16_24.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22230622_10-04-2025_16_16_24.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','22230622','results_Auto_P22230622_10-04-2025_16_16_24.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22230622_10-04-2025_16_16_24.pdf','pdf','','0.49','0.0887619702105379') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/22230622

https://www.fotonower.com/image?json=false&list_photos_id=1351168258
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1351168255
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1351168240
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1351168237
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1351168234
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1351168220
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1351168204
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 8.88%
Veuillez trouver les photos des contaminants.

exemples de contaminants: carton: https://www.fotonower.com/view/22231443?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/22231447?limit=200
exemples de contaminants: metal: https://www.fotonower.com/view/22231450?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/22231452?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22230622_10-04-2025_16_16_24.pdf.

Lien vers velours :https://www.fotonower.com/velours/22231443,22231444,22231445,22231446,22231447,22231448,22231449,22231450,22231451,22231452,22231453?tags=carton,pet_fonce,flou,autre,papier,background,mal_croppe,metal,pehd,pet_clair,environnement.


L'équipe Fotonower 202 b'' Server: nginx Date: Thu, 10 Apr 2025 14:16:34 GMT Content-Length: 0 Connection: close X-Message-Id: 2_gHtsDLTF2OzBYpqEIg9w Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1351168258, 1351168255, 1351168240, 1351168237, 1351168234, 1351168220, 1351168204] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168258', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168255', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168240', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168237', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168234', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168220', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168204', None, None, None, None, None, '2740212') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 7 time used for this insertion : 0.01325225830078125 save_final save missing photos in datou_result : time spend for datou_step_exec : 9.474462747573853 time spend to save output : 0.013457536697387695 total time spend for step 9 : 9.48792028427124 step10:split_time_score Thu Apr 10 16:16:34 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('14', 6), ('15', 1)) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 10042025 22230622 Nombre de photos uploadées : 7 / 23040 (0%) 10042025 22230622 Nombre de photos taguées (types de déchets): 0 / 7 (0%) 10042025 22230622 Nombre de photos taguées (volume) : 0 / 7 (0%) elapsed_time : load_data_split_time_score 1.6689300537109375e-06 elapsed_time : order_list_meta_photo_and_scores 5.0067901611328125e-06 ??????? elapsed_time : fill_and_build_computed_from_old_data 0.0004172325134277344 elapsed_time : insert_dashboard_record_day_entry 0.02651834487915039 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.0382817973935593 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22224039_10-04-2025_13_45_17.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22224039 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22224039 AND mptpi.`type`=3726 To do Qualite : 0.03490860759357629 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22223606_10-04-2025_14_09_34.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22223606 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22223606 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22225864 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22225872 order by id desc limit 1 Qualite : 0.0887619702105379 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22230622_10-04-2025_16_16_24.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22230622 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22230622 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22230624 order by id desc limit 1 NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'10042025': {'nb_upload': 7, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1351168258, 1351168255, 1351168240, 1351168237, 1351168234, 1351168220, 1351168204] Looping around the photos to save general results len do output : 1 /22230622Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168258', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168255', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168240', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168237', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168234', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168220', None, None, None, None, None, '2740212') ('3318', None, None, None, None, None, None, None, '2740212') ('3318', '22230622', '1351168204', None, None, None, None, None, '2740212') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 8 time used for this insertion : 0.20225119590759277 save_final save missing photos in datou_result : time spend for datou_step_exec : 3.037191152572632 time spend to save output : 0.2024979591369629 total time spend for step 10 : 3.2396891117095947 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 7 set_done_treatment 117.23user 133.99system 6:11.88elapsed 67%CPU (0avgtext+0avgdata 4531228maxresident)k 688712inputs+88088outputs (10617major+13224655minor)pagefaults 0swaps