python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2111745 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['2893741'] with mtr_portfolio_ids : ['23118015'] and first list_photo_ids : [] new path : /proc/2111745/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 6 ; length of list_pids : 6 ; length of list_args : 6 time to download the photos : 1.0825884342193604 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Mon May 19 14:40:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10593 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-05-19 14:40:31.509266: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-05-19 14:40:31.535149: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-05-19 14:40:31.537279: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f2360000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-05-19 14:40:31.537321: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-05-19 14:40:31.541003: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-05-19 14:40:31.777583: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x28b91bb0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-05-19 14:40:31.777623: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-05-19 14:40:31.779038: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-19 14:40:31.779430: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-19 14:40:31.782197: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-19 14:40:31.784776: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-19 14:40:31.785248: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-19 14:40:31.788183: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-19 14:40:31.789178: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-19 14:40:31.793249: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-19 14:40:31.794596: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-19 14:40:31.794653: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-19 14:40:31.795438: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-19 14:40:31.795455: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-19 14:40:31.795464: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-19 14:40:31.796834: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9815 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-05-19 14:40:32.069489: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-19 14:40:32.069584: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-19 14:40:32.069608: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-19 14:40:32.069630: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-19 14:40:32.069651: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-19 14:40:32.069672: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-19 14:40:32.069692: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-19 14:40:32.069713: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-19 14:40:32.071347: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-19 14:40:32.072722: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-19 14:40:32.072758: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-19 14:40:32.072778: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-19 14:40:32.072797: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-19 14:40:32.072815: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-19 14:40:32.072833: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-19 14:40:32.072851: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-19 14:40:32.072868: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-19 14:40:32.074496: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-19 14:40:32.074526: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-19 14:40:32.074536: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-19 14:40:32.074545: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-19 14:40:32.076207: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9815 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-05-19 14:40:43.485087: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-19 14:40:43.710608: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 6 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 3.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 29 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 32 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 22 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 6.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 51 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 11.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 29 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 15.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 22 Detection mask done ! Trying to reset tf kernel 2112486 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 334 tf kernel not reseted sub process len(results) : 6 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 6 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 6816 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0063779354095458984 nb_pixel_total : 243895 time to create 1 rle with new method : 0.014159202575683594 length of segment : 789 time for calcul the mask position with numpy : 0.00240325927734375 nb_pixel_total : 86894 time to create 1 rle with old method : 0.116943359375 length of segment : 778 time for calcul the mask position with numpy : 0.011758565902709961 nb_pixel_total : 271459 time to create 1 rle with new method : 0.015436172485351562 length of segment : 488 time for calcul the mask position with numpy : 0.0007722377777099609 nb_pixel_total : 23156 time to create 1 rle with old method : 0.02681756019592285 length of segment : 200 time for calcul the mask position with numpy : 0.0021250247955322266 nb_pixel_total : 96069 time to create 1 rle with old method : 0.11257052421569824 length of segment : 453 time for calcul the mask position with numpy : 0.0014164447784423828 nb_pixel_total : 59704 time to create 1 rle with old method : 0.06818842887878418 length of segment : 265 time for calcul the mask position with numpy : 0.0004286766052246094 nb_pixel_total : 17213 time to create 1 rle with old method : 0.019398927688598633 length of segment : 169 time for calcul the mask position with numpy : 0.00043654441833496094 nb_pixel_total : 14375 time to create 1 rle with old method : 0.016064167022705078 length of segment : 156 time for calcul the mask position with numpy : 0.0031943321228027344 nb_pixel_total : 176596 time to create 1 rle with new method : 0.011618852615356445 length of segment : 513 time for calcul the mask position with numpy : 0.0012280941009521484 nb_pixel_total : 43688 time to create 1 rle with old method : 0.05188322067260742 length of segment : 283 time for calcul the mask position with numpy : 0.003186464309692383 nb_pixel_total : 160169 time to create 1 rle with new method : 0.01020193099975586 length of segment : 390 time for calcul the mask position with numpy : 0.00021314620971679688 nb_pixel_total : 8549 time to create 1 rle with old method : 0.010178565979003906 length of segment : 117 time for calcul the mask position with numpy : 0.0004329681396484375 nb_pixel_total : 17278 time to create 1 rle with old method : 0.020310640335083008 length of segment : 286 time for calcul the mask position with numpy : 0.0021026134490966797 nb_pixel_total : 137403 time to create 1 rle with old method : 0.15923810005187988 length of segment : 614 time for calcul the mask position with numpy : 0.0003249645233154297 nb_pixel_total : 13220 time to create 1 rle with old method : 0.015574932098388672 length of segment : 102 time for calcul the mask position with numpy : 0.0009441375732421875 nb_pixel_total : 47260 time to create 1 rle with old method : 0.09141254425048828 length of segment : 385 time for calcul the mask position with numpy : 0.00064849853515625 nb_pixel_total : 22423 time to create 1 rle with old method : 0.02570486068725586 length of segment : 287 time for calcul the mask position with numpy : 0.0005021095275878906 nb_pixel_total : 28429 time to create 1 rle with old method : 0.032874345779418945 length of segment : 134 time for calcul the mask position with numpy : 0.0005698204040527344 nb_pixel_total : 13607 time to create 1 rle with old method : 0.01612114906311035 length of segment : 154 time for calcul the mask position with numpy : 0.0027909278869628906 nb_pixel_total : 150194 time to create 1 rle with new method : 0.010019540786743164 length of segment : 351 time for calcul the mask position with numpy : 0.0005242824554443359 nb_pixel_total : 17213 time to create 1 rle with old method : 0.020267009735107422 length of segment : 178 time for calcul the mask position with numpy : 0.0016248226165771484 nb_pixel_total : 38463 time to create 1 rle with old method : 0.05250906944274902 length of segment : 209 time for calcul the mask position with numpy : 0.0405731201171875 nb_pixel_total : 330218 time to create 1 rle with new method : 0.10571670532226562 length of segment : 940 time for calcul the mask position with numpy : 0.0005233287811279297 nb_pixel_total : 3244 time to create 1 rle with old method : 0.004392862319946289 length of segment : 90 time for calcul the mask position with numpy : 0.001134634017944336 nb_pixel_total : 60190 time to create 1 rle with old method : 0.07067728042602539 length of segment : 244 time for calcul the mask position with numpy : 0.0016646385192871094 nb_pixel_total : 22987 time to create 1 rle with old method : 0.03767037391662598 length of segment : 237 time for calcul the mask position with numpy : 0.0009093284606933594 nb_pixel_total : 12502 time to create 1 rle with old method : 0.021268367767333984 length of segment : 167 time for calcul the mask position with numpy : 0.0011653900146484375 nb_pixel_total : 17087 time to create 1 rle with old method : 0.028539419174194336 length of segment : 161 time for calcul the mask position with numpy : 0.00999593734741211 nb_pixel_total : 181919 time to create 1 rle with new method : 0.01700758934020996 length of segment : 739 time for calcul the mask position with numpy : 0.001001119613647461 nb_pixel_total : 15587 time to create 1 rle with old method : 0.01852273941040039 length of segment : 211 time for calcul the mask position with numpy : 0.0024690628051757812 nb_pixel_total : 39350 time to create 1 rle with old method : 0.04793739318847656 length of segment : 522 time for calcul the mask position with numpy : 0.0013031959533691406 nb_pixel_total : 49792 time to create 1 rle with old method : 0.05996584892272949 length of segment : 217 time for calcul the mask position with numpy : 0.0007495880126953125 nb_pixel_total : 21662 time to create 1 rle with old method : 0.02583003044128418 length of segment : 177 time for calcul the mask position with numpy : 0.0021865367889404297 nb_pixel_total : 81842 time to create 1 rle with old method : 0.10149121284484863 length of segment : 326 time for calcul the mask position with numpy : 0.0007026195526123047 nb_pixel_total : 23015 time to create 1 rle with old method : 0.02733016014099121 length of segment : 129 time for calcul the mask position with numpy : 0.0012466907501220703 nb_pixel_total : 28332 time to create 1 rle with old method : 0.033184051513671875 length of segment : 252 time for calcul the mask position with numpy : 0.0004677772521972656 nb_pixel_total : 8529 time to create 1 rle with old method : 0.010391950607299805 length of segment : 131 time for calcul the mask position with numpy : 0.001718759536743164 nb_pixel_total : 42369 time to create 1 rle with old method : 0.04882168769836426 length of segment : 265 time for calcul the mask position with numpy : 0.0004565715789794922 nb_pixel_total : 11790 time to create 1 rle with old method : 0.014145612716674805 length of segment : 103 time for calcul the mask position with numpy : 0.0009279251098632812 nb_pixel_total : 40346 time to create 1 rle with old method : 0.048488616943359375 length of segment : 224 time for calcul the mask position with numpy : 0.0010833740234375 nb_pixel_total : 31582 time to create 1 rle with old method : 0.03544330596923828 length of segment : 250 time for calcul the mask position with numpy : 0.000354766845703125 nb_pixel_total : 12764 time to create 1 rle with old method : 0.015016555786132812 length of segment : 103 time for calcul the mask position with numpy : 0.00042366981506347656 nb_pixel_total : 12161 time to create 1 rle with old method : 0.01448369026184082 length of segment : 140 time for calcul the mask position with numpy : 0.0003998279571533203 nb_pixel_total : 11457 time to create 1 rle with old method : 0.014080524444580078 length of segment : 71 time for calcul the mask position with numpy : 0.00074005126953125 nb_pixel_total : 19510 time to create 1 rle with old method : 0.023054838180541992 length of segment : 171 time for calcul the mask position with numpy : 0.0006601810455322266 nb_pixel_total : 17982 time to create 1 rle with old method : 0.021058082580566406 length of segment : 251 time for calcul the mask position with numpy : 0.0011870861053466797 nb_pixel_total : 26410 time to create 1 rle with old method : 0.030747652053833008 length of segment : 281 time for calcul the mask position with numpy : 0.0018718242645263672 nb_pixel_total : 37501 time to create 1 rle with old method : 0.0427703857421875 length of segment : 370 time for calcul the mask position with numpy : 0.0009582042694091797 nb_pixel_total : 17393 time to create 1 rle with old method : 0.020104169845581055 length of segment : 123 time for calcul the mask position with numpy : 0.0032339096069335938 nb_pixel_total : 58161 time to create 1 rle with old method : 0.06630420684814453 length of segment : 361 time for calcul the mask position with numpy : 0.0021271705627441406 nb_pixel_total : 34896 time to create 1 rle with old method : 0.040773868560791016 length of segment : 306 time for calcul the mask position with numpy : 0.0011780261993408203 nb_pixel_total : 16912 time to create 1 rle with old method : 0.020165205001831055 length of segment : 221 time for calcul the mask position with numpy : 0.005461692810058594 nb_pixel_total : 108911 time to create 1 rle with old method : 0.12223434448242188 length of segment : 511 time for calcul the mask position with numpy : 0.001543283462524414 nb_pixel_total : 18887 time to create 1 rle with old method : 0.022391796112060547 length of segment : 265 time for calcul the mask position with numpy : 0.004885673522949219 nb_pixel_total : 126023 time to create 1 rle with old method : 0.1424102783203125 length of segment : 454 time for calcul the mask position with numpy : 0.0017838478088378906 nb_pixel_total : 37702 time to create 1 rle with old method : 0.04455828666687012 length of segment : 232 time for calcul the mask position with numpy : 0.004563093185424805 nb_pixel_total : 67257 time to create 1 rle with old method : 0.07732701301574707 length of segment : 433 time for calcul the mask position with numpy : 0.0006353855133056641 nb_pixel_total : 7384 time to create 1 rle with old method : 0.012256860733032227 length of segment : 183 time for calcul the mask position with numpy : 0.0006117820739746094 nb_pixel_total : 12152 time to create 1 rle with old method : 0.02069544792175293 length of segment : 105 time for calcul the mask position with numpy : 0.004819631576538086 nb_pixel_total : 69674 time to create 1 rle with old method : 0.08153581619262695 length of segment : 331 time for calcul the mask position with numpy : 0.0012366771697998047 nb_pixel_total : 25786 time to create 1 rle with old method : 0.03003215789794922 length of segment : 238 time for calcul the mask position with numpy : 0.0018482208251953125 nb_pixel_total : 50721 time to create 1 rle with old method : 0.05818319320678711 length of segment : 210 time for calcul the mask position with numpy : 0.002252340316772461 nb_pixel_total : 54646 time to create 1 rle with old method : 0.06316947937011719 length of segment : 262 time for calcul the mask position with numpy : 0.0007898807525634766 nb_pixel_total : 9327 time to create 1 rle with old method : 0.016002416610717773 length of segment : 108 time for calcul the mask position with numpy : 0.01201176643371582 nb_pixel_total : 235979 time to create 1 rle with new method : 0.019437789916992188 length of segment : 538 time for calcul the mask position with numpy : 0.0016956329345703125 nb_pixel_total : 36829 time to create 1 rle with old method : 0.04220843315124512 length of segment : 289 time for calcul the mask position with numpy : 0.0015392303466796875 nb_pixel_total : 42394 time to create 1 rle with old method : 0.052593231201171875 length of segment : 442 time for calcul the mask position with numpy : 0.0010235309600830078 nb_pixel_total : 14637 time to create 1 rle with old method : 0.01720285415649414 length of segment : 172 time for calcul the mask position with numpy : 0.0039463043212890625 nb_pixel_total : 43110 time to create 1 rle with old method : 0.05433249473571777 length of segment : 674 time for calcul the mask position with numpy : 0.004320859909057617 nb_pixel_total : 44232 time to create 1 rle with old method : 0.052643537521362305 length of segment : 471 time for calcul the mask position with numpy : 0.0015327930450439453 nb_pixel_total : 30852 time to create 1 rle with old method : 0.036171913146972656 length of segment : 286 time for calcul the mask position with numpy : 0.0012516975402832031 nb_pixel_total : 16872 time to create 1 rle with old method : 0.019617319107055664 length of segment : 281 time for calcul the mask position with numpy : 0.0013875961303710938 nb_pixel_total : 19499 time to create 1 rle with old method : 0.023199796676635742 length of segment : 288 time for calcul the mask position with numpy : 0.003263235092163086 nb_pixel_total : 36328 time to create 1 rle with old method : 0.04345393180847168 length of segment : 372 time for calcul the mask position with numpy : 0.0036699771881103516 nb_pixel_total : 61601 time to create 1 rle with old method : 0.0790410041809082 length of segment : 629 time for calcul the mask position with numpy : 0.0016655921936035156 nb_pixel_total : 38013 time to create 1 rle with old method : 0.04714560508728027 length of segment : 340 time for calcul the mask position with numpy : 0.0027205944061279297 nb_pixel_total : 71590 time to create 1 rle with old method : 0.08207297325134277 length of segment : 358 time for calcul the mask position with numpy : 0.0012116432189941406 nb_pixel_total : 33607 time to create 1 rle with old method : 0.03937101364135742 length of segment : 226 time for calcul the mask position with numpy : 0.004157066345214844 nb_pixel_total : 144009 time to create 1 rle with old method : 0.16503429412841797 length of segment : 305 time for calcul the mask position with numpy : 0.0008523464202880859 nb_pixel_total : 20296 time to create 1 rle with old method : 0.02384495735168457 length of segment : 186 time for calcul the mask position with numpy : 0.0021352767944335938 nb_pixel_total : 53375 time to create 1 rle with old method : 0.0618433952331543 length of segment : 358 time for calcul the mask position with numpy : 0.0016155242919921875 nb_pixel_total : 28730 time to create 1 rle with old method : 0.03448987007141113 length of segment : 218 time for calcul the mask position with numpy : 0.0002968311309814453 nb_pixel_total : 8273 time to create 1 rle with old method : 0.010051727294921875 length of segment : 95 time for calcul the mask position with numpy : 0.0017669200897216797 nb_pixel_total : 77260 time to create 1 rle with old method : 0.09016704559326172 length of segment : 378 time for calcul the mask position with numpy : 0.0017867088317871094 nb_pixel_total : 49876 time to create 1 rle with old method : 0.05917024612426758 length of segment : 279 time for calcul the mask position with numpy : 0.006725311279296875 nb_pixel_total : 113673 time to create 1 rle with old method : 0.13160371780395508 length of segment : 605 time for calcul the mask position with numpy : 0.0014226436614990234 nb_pixel_total : 31617 time to create 1 rle with old method : 0.037328481674194336 length of segment : 472 time for calcul the mask position with numpy : 0.0003676414489746094 nb_pixel_total : 9745 time to create 1 rle with old method : 0.011911392211914062 length of segment : 69 time for calcul the mask position with numpy : 0.00016117095947265625 nb_pixel_total : 8317 time to create 1 rle with old method : 0.010130643844604492 length of segment : 94 time for calcul the mask position with numpy : 0.0011119842529296875 nb_pixel_total : 64450 time to create 1 rle with old method : 0.0737924575805664 length of segment : 372 time for calcul the mask position with numpy : 0.0007939338684082031 nb_pixel_total : 25376 time to create 1 rle with old method : 0.030027389526367188 length of segment : 251 time for calcul the mask position with numpy : 0.0013890266418457031 nb_pixel_total : 40405 time to create 1 rle with old method : 0.04732322692871094 length of segment : 291 time for calcul the mask position with numpy : 0.0003540515899658203 nb_pixel_total : 9525 time to create 1 rle with old method : 0.012249231338500977 length of segment : 96 time for calcul the mask position with numpy : 0.00042724609375 nb_pixel_total : 12773 time to create 1 rle with old method : 0.014946937561035156 length of segment : 116 time for calcul the mask position with numpy : 0.0004019737243652344 nb_pixel_total : 13014 time to create 1 rle with old method : 0.01556706428527832 length of segment : 152 time for calcul the mask position with numpy : 0.0005049705505371094 nb_pixel_total : 11761 time to create 1 rle with old method : 0.013727664947509766 length of segment : 126 time for calcul the mask position with numpy : 0.0019481182098388672 nb_pixel_total : 38223 time to create 1 rle with old method : 0.04563570022583008 length of segment : 217 time for calcul the mask position with numpy : 0.23357796669006348 nb_pixel_total : 4109419 time to create 1 rle with new method : 0.7537474632263184 length of segment : 2337 time for calcul the mask position with numpy : 0.0005536079406738281 nb_pixel_total : 19983 time to create 1 rle with old method : 0.02406597137451172 length of segment : 216 time for calcul the mask position with numpy : 0.0006353855133056641 nb_pixel_total : 39632 time to create 1 rle with old method : 0.04682183265686035 length of segment : 174 time for calcul the mask position with numpy : 0.00037384033203125 nb_pixel_total : 14640 time to create 1 rle with old method : 0.0186765193939209 length of segment : 140 time for calcul the mask position with numpy : 0.0005390644073486328 nb_pixel_total : 26637 time to create 1 rle with old method : 0.03360795974731445 length of segment : 254 time for calcul the mask position with numpy : 0.00034332275390625 nb_pixel_total : 21277 time to create 1 rle with old method : 0.026613712310791016 length of segment : 92 time for calcul the mask position with numpy : 0.0018072128295898438 nb_pixel_total : 129906 time to create 1 rle with old method : 0.15408754348754883 length of segment : 242 time for calcul the mask position with numpy : 0.0003693103790283203 nb_pixel_total : 7384 time to create 1 rle with old method : 0.008803129196166992 length of segment : 106 time spent for convertir_results : 12.502183437347412 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 105 chid ids of type : 3594 Number RLEs to save : 32023 save missing photos in datou_result : time spend for datou_step_exec : 66.00867486000061 time spend to save output : 1.8604762554168701 total time spend for step 1 : 67.86915111541748 step2:crop_condition Mon May 19 14:41:36 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 6 ! batch 1 Loaded 105 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 81 About to insert : list_path_to_insert length 81 new photo from crops ! About to upload 81 photos upload in portfolio : 3736932 init cache_photo without model_param we have 81 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747658518_2111745 we have uploaded 81 photos in the portfolio 3736932 time of upload the photos Elapsed time : 22.346277952194214 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 15 About to insert : list_path_to_insert length 15 new photo from crops ! About to upload 15 photos upload in portfolio : 3736932 init cache_photo without model_param we have 15 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747658546_2111745 we have uploaded 15 photos in the portfolio 3736932 time of upload the photos Elapsed time : 4.499250173568726 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3736932 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747658552_2111745 we have uploaded 2 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.9873931407928467 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! About to upload 7 photos upload in portfolio : 3736932 init cache_photo without model_param we have 7 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747658574_2111745 we have uploaded 7 photos in the portfolio 3736932 time of upload the photos Elapsed time : 2.4640207290649414 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1359350759, 1359350600, 1359350584, 1359350569, 1359350550, 1359350520] Looping around the photos to save general results len do output : 105 /1359377296Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377297Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377298Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377299Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377300Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377301Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377302Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377303Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377304Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377305Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377306Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377307Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377308Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377309Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377310Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377311Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377312Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377313Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377314Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377315Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377316Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377317Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377318Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377319Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377320Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377321Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377322Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377323Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377324Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377325Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377326Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377327Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377328Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377329Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377330Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377331Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377332Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377334Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377335Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377336Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377338Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377339Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377340Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377343Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377344Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377345Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377346Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377348Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377349Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377350Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377352Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377353Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377354Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377356Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377357Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377358Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377360Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377361Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377362Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377363Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377365Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377366Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377367Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377369Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377370Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377371Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377373Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377374Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377375Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377377Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377378Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377379Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377381Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377382Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377383Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377384Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377386Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377387Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377388Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377389Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377390Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377402Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377403Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377405Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377406Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377408Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377410Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377411Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377412Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377414Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377415Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377417Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377418Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377420Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377421Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377422Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377441Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377442Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377498Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377499Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377500Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377501Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377502Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377503Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359377504Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350759', None, None, None, None, None, '2893741') ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350600', None, None, None, None, None, '2893741') ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350584', None, None, None, None, None, '2893741') ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350569', None, None, None, None, None, '2893741') ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350550', None, None, None, None, None, '2893741') ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350520', None, None, None, None, None, '2893741') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 321 time used for this insertion : 0.027303218841552734 save_final save missing photos in datou_result : time spend for datou_step_exec : 80.10414266586304 time spend to save output : 0.031261444091796875 total time spend for step 2 : 80.13540410995483 step3:rle_unique_nms_with_priority Mon May 19 14:42:56 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 105 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 10 nb_hashtags : 2 time to prepare the origin masks : 5.9856531620025635 time for calcul the mask position with numpy : 0.8227050304412842 nb_pixel_total : 7261579 time to create 1 rle with new method : 0.690638542175293 time for calcul the mask position with numpy : 0.024585247039794922 nb_pixel_total : 43460 time to create 1 rle with old method : 0.048397064208984375 time for calcul the mask position with numpy : 0.026492834091186523 nb_pixel_total : 176596 time to create 1 rle with new method : 1.0861930847167969 time for calcul the mask position with numpy : 0.02592945098876953 nb_pixel_total : 14375 time to create 1 rle with old method : 0.016213417053222656 time for calcul the mask position with numpy : 0.02523040771484375 nb_pixel_total : 17213 time to create 1 rle with old method : 0.01938796043395996 time for calcul the mask position with numpy : 0.02535080909729004 nb_pixel_total : 59704 time to create 1 rle with old method : 0.06681680679321289 time for calcul the mask position with numpy : 0.024480342864990234 nb_pixel_total : 96069 time to create 1 rle with old method : 0.10654115676879883 time for calcul the mask position with numpy : 0.024655818939208984 nb_pixel_total : 23156 time to create 1 rle with old method : 0.02594900131225586 time for calcul the mask position with numpy : 0.025634050369262695 nb_pixel_total : 271459 time to create 1 rle with new method : 0.9419994354248047 time for calcul the mask position with numpy : 0.02544879913330078 nb_pixel_total : 86894 time to create 1 rle with old method : 0.10105752944946289 time for calcul the mask position with numpy : 0.028110265731811523 nb_pixel_total : 243895 time to create 1 rle with new method : 1.112992525100708 create new chi : 5.419402837753296 time to delete rle : 0.031132221221923828 batch 1 Loaded 21 chid ids of type : 3594 ++++++++++++++Number RLEs to save : 10244 TO DO : save crop sub photo not yet done ! save time : 0.7101798057556152 nb_obj : 21 nb_hashtags : 2 time to prepare the origin masks : 10.216195821762085 time for calcul the mask position with numpy : 0.7046427726745605 nb_pixel_total : 7016519 time to create 1 rle with new method : 2.2330994606018066 time for calcul the mask position with numpy : 0.03547525405883789 nb_pixel_total : 39350 time to create 1 rle with old method : 0.04429364204406738 time for calcul the mask position with numpy : 0.02417755126953125 nb_pixel_total : 15587 time to create 1 rle with old method : 0.017838239669799805 time for calcul the mask position with numpy : 0.028995990753173828 nb_pixel_total : 178079 time to create 1 rle with new method : 0.6813485622406006 time for calcul the mask position with numpy : 0.025159358978271484 nb_pixel_total : 15271 time to create 1 rle with old method : 0.017401933670043945 time for calcul the mask position with numpy : 0.023998022079467773 nb_pixel_total : 12502 time to create 1 rle with old method : 0.01391148567199707 time for calcul the mask position with numpy : 0.02472090721130371 nb_pixel_total : 22840 time to create 1 rle with old method : 0.02493429183959961 time for calcul the mask position with numpy : 0.025107622146606445 nb_pixel_total : 6582 time to create 1 rle with old method : 0.007259845733642578 time for calcul the mask position with numpy : 0.024653196334838867 nb_pixel_total : 3244 time to create 1 rle with old method : 0.0036101341247558594 time for calcul the mask position with numpy : 0.029234886169433594 nb_pixel_total : 330218 time to create 1 rle with new method : 1.7666196823120117 time for calcul the mask position with numpy : 0.027796268463134766 nb_pixel_total : 38463 time to create 1 rle with old method : 0.0479581356048584 time for calcul the mask position with numpy : 0.027162551879882812 nb_pixel_total : 17213 time to create 1 rle with old method : 0.01941537857055664 time for calcul the mask position with numpy : 0.030778169631958008 nb_pixel_total : 150194 time to create 1 rle with new method : 1.1527822017669678 time for calcul the mask position with numpy : 0.025034427642822266 nb_pixel_total : 13607 time to create 1 rle with old method : 0.014564990997314453 time for calcul the mask position with numpy : 0.02505016326904297 nb_pixel_total : 28429 time to create 1 rle with old method : 0.030661582946777344 time for calcul the mask position with numpy : 0.0240170955657959 nb_pixel_total : 22423 time to create 1 rle with old method : 0.024088621139526367 time for calcul the mask position with numpy : 0.024688243865966797 nb_pixel_total : 47260 time to create 1 rle with old method : 0.05212569236755371 time for calcul the mask position with numpy : 0.024573326110839844 nb_pixel_total : 13220 time to create 1 rle with old method : 0.01614546775817871 time for calcul the mask position with numpy : 0.024384498596191406 nb_pixel_total : 137403 time to create 1 rle with old method : 0.15658831596374512 time for calcul the mask position with numpy : 0.024476289749145508 nb_pixel_total : 17278 time to create 1 rle with old method : 0.022554636001586914 time for calcul the mask position with numpy : 0.026590824127197266 nb_pixel_total : 8549 time to create 1 rle with old method : 0.010895490646362305 time for calcul the mask position with numpy : 0.042342185974121094 nb_pixel_total : 160169 time to create 1 rle with new method : 0.8449749946594238 create new chi : 8.630436420440674 time to delete rle : 0.001989126205444336 batch 1 Loaded 43 chid ids of type : 3594 +++++++++++++++++++++++++++++++Number RLEs to save : 14564 TO DO : save crop sub photo not yet done ! save time : 1.0998468399047852 nb_obj : 16 nb_hashtags : 3 time to prepare the origin masks : 11.535495042800903 time for calcul the mask position with numpy : 0.530555009841919 nb_pixel_total : 7854857 time to create 1 rle with new method : 1.700225591659546 time for calcul the mask position with numpy : 0.041634321212768555 nb_pixel_total : 26410 time to create 1 rle with old method : 0.029814958572387695 time for calcul the mask position with numpy : 0.025084972381591797 nb_pixel_total : 17982 time to create 1 rle with old method : 0.020285367965698242 time for calcul the mask position with numpy : 0.024943113327026367 nb_pixel_total : 19510 time to create 1 rle with old method : 0.021911144256591797 time for calcul the mask position with numpy : 0.024941682815551758 nb_pixel_total : 11457 time to create 1 rle with old method : 0.012867450714111328 time for calcul the mask position with numpy : 0.0292510986328125 nb_pixel_total : 12161 time to create 1 rle with old method : 0.013626813888549805 time for calcul the mask position with numpy : 0.026626110076904297 nb_pixel_total : 12764 time to create 1 rle with old method : 0.01441192626953125 time for calcul the mask position with numpy : 0.026511669158935547 nb_pixel_total : 31582 time to create 1 rle with old method : 0.03806328773498535 time for calcul the mask position with numpy : 0.037358999252319336 nb_pixel_total : 40346 time to create 1 rle with old method : 0.04589581489562988 time for calcul the mask position with numpy : 0.04115557670593262 nb_pixel_total : 11790 time to create 1 rle with old method : 0.013345003128051758 time for calcul the mask position with numpy : 0.04244661331176758 nb_pixel_total : 42369 time to create 1 rle with old method : 0.05361175537109375 time for calcul the mask position with numpy : 0.04339766502380371 nb_pixel_total : 8529 time to create 1 rle with old method : 0.009737014770507812 time for calcul the mask position with numpy : 0.04182577133178711 nb_pixel_total : 28332 time to create 1 rle with old method : 0.03138399124145508 time for calcul the mask position with numpy : 0.040651559829711914 nb_pixel_total : 23015 time to create 1 rle with old method : 0.026128292083740234 time for calcul the mask position with numpy : 0.04026436805725098 nb_pixel_total : 81842 time to create 1 rle with old method : 0.0907750129699707 time for calcul the mask position with numpy : 0.04023385047912598 nb_pixel_total : 21662 time to create 1 rle with old method : 0.024554014205932617 time for calcul the mask position with numpy : 0.04087710380554199 nb_pixel_total : 49792 time to create 1 rle with old method : 0.060254812240600586 create new chi : 3.3533287048339844 time to delete rle : 0.001291513442993164 batch 1 Loaded 33 chid ids of type : 3594 +++++++++++++++++Number RLEs to save : 8342 TO DO : save crop sub photo not yet done ! save time : 0.5635030269622803 nb_obj : 28 nb_hashtags : 3 time to prepare the origin masks : 4.615358352661133 time for calcul the mask position with numpy : 1.560279130935669 nb_pixel_total : 6981382 time to create 1 rle with new method : 0.7362651824951172 time for calcul the mask position with numpy : 0.0332026481628418 nb_pixel_total : 30852 time to create 1 rle with old method : 0.034052133560180664 time for calcul the mask position with numpy : 0.0328519344329834 nb_pixel_total : 37501 time to create 1 rle with old method : 0.040807485580444336 time for calcul the mask position with numpy : 0.0338592529296875 nb_pixel_total : 23803 time to create 1 rle with old method : 0.02769184112548828 time for calcul the mask position with numpy : 0.03398942947387695 nb_pixel_total : 16912 time to create 1 rle with old method : 0.019456148147583008 time for calcul the mask position with numpy : 0.03537464141845703 nb_pixel_total : 54646 time to create 1 rle with old method : 0.06156110763549805 time for calcul the mask position with numpy : 0.03427767753601074 nb_pixel_total : 58161 time to create 1 rle with old method : 0.06741213798522949 time for calcul the mask position with numpy : 0.03496599197387695 nb_pixel_total : 69674 time to create 1 rle with old method : 0.07761192321777344 time for calcul the mask position with numpy : 0.032662391662597656 nb_pixel_total : 34896 time to create 1 rle with old method : 0.037286996841430664 time for calcul the mask position with numpy : 0.03281831741333008 nb_pixel_total : 7384 time to create 1 rle with old method : 0.008067607879638672 time for calcul the mask position with numpy : 0.03345608711242676 nb_pixel_total : 43110 time to create 1 rle with old method : 0.04690837860107422 time for calcul the mask position with numpy : 0.033652544021606445 nb_pixel_total : 50721 time to create 1 rle with old method : 0.05624794960021973 time for calcul the mask position with numpy : 0.03335905075073242 nb_pixel_total : 14637 time to create 1 rle with old method : 0.016109704971313477 time for calcul the mask position with numpy : 0.0339205265045166 nb_pixel_total : 44232 time to create 1 rle with old method : 0.048880577087402344 time for calcul the mask position with numpy : 0.03349018096923828 nb_pixel_total : 67257 time to create 1 rle with old method : 0.07980561256408691 time for calcul the mask position with numpy : 0.037232398986816406 nb_pixel_total : 36829 time to create 1 rle with old method : 0.03954792022705078 time for calcul the mask position with numpy : 0.03206157684326172 nb_pixel_total : 61601 time to create 1 rle with old method : 0.06647324562072754 time for calcul the mask position with numpy : 0.033071041107177734 nb_pixel_total : 37702 time to create 1 rle with old method : 0.04099106788635254 time for calcul the mask position with numpy : 0.033471107482910156 nb_pixel_total : 108911 time to create 1 rle with old method : 0.11587834358215332 time for calcul the mask position with numpy : 0.033129215240478516 nb_pixel_total : 18887 time to create 1 rle with old method : 0.021013975143432617 time for calcul the mask position with numpy : 0.03668475151062012 nb_pixel_total : 230171 time to create 1 rle with new method : 2.097769021987915 time for calcul the mask position with numpy : 0.03369021415710449 nb_pixel_total : 17393 time to create 1 rle with old method : 0.01904153823852539 time for calcul the mask position with numpy : 0.03322887420654297 nb_pixel_total : 126023 time to create 1 rle with old method : 0.15198087692260742 time for calcul the mask position with numpy : 0.03507375717163086 nb_pixel_total : 41070 time to create 1 rle with old method : 0.04625654220581055 time for calcul the mask position with numpy : 0.034300804138183594 nb_pixel_total : 19499 time to create 1 rle with old method : 0.022072792053222656 time for calcul the mask position with numpy : 0.03536224365234375 nb_pixel_total : 36058 time to create 1 rle with old method : 0.043462514877319336 time for calcul the mask position with numpy : 0.03628087043762207 nb_pixel_total : 3609 time to create 1 rle with old method : 0.004153251647949219 time for calcul the mask position with numpy : 0.034331321716308594 nb_pixel_total : 9327 time to create 1 rle with old method : 0.010721206665039062 time for calcul the mask position with numpy : 0.03424549102783203 nb_pixel_total : 12152 time to create 1 rle with old method : 0.013717889785766602 create new chi : 6.633347272872925 time to delete rle : 0.002440214157104492 batch 1 Loaded 57 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 19683 TO DO : save crop sub photo not yet done ! save time : 1.3030750751495361 nb_obj : 16 nb_hashtags : 2 time to prepare the origin masks : 10.217714309692383 time for calcul the mask position with numpy : 0.5653941631317139 nb_pixel_total : 7564073 time to create 1 rle with new method : 0.6905946731567383 time for calcul the mask position with numpy : 0.027181386947631836 nb_pixel_total : 25318 time to create 1 rle with old method : 0.028847455978393555 time for calcul the mask position with numpy : 0.026625633239746094 nb_pixel_total : 28747 time to create 1 rle with old method : 0.04654669761657715 time for calcul the mask position with numpy : 0.026965856552124023 nb_pixel_total : 19 time to create 1 rle with old method : 6.461143493652344e-05 time for calcul the mask position with numpy : 0.029148101806640625 nb_pixel_total : 9745 time to create 1 rle with old method : 0.016125202178955078 time for calcul the mask position with numpy : 0.02823162078857422 nb_pixel_total : 31617 time to create 1 rle with old method : 0.047333478927612305 time for calcul the mask position with numpy : 0.026894092559814453 nb_pixel_total : 109852 time to create 1 rle with old method : 0.1322486400604248 time for calcul the mask position with numpy : 0.025417566299438477 nb_pixel_total : 49876 time to create 1 rle with old method : 0.05569934844970703 time for calcul the mask position with numpy : 0.02748274803161621 nb_pixel_total : 77260 time to create 1 rle with old method : 0.08738446235656738 time for calcul the mask position with numpy : 0.03399467468261719 nb_pixel_total : 8273 time to create 1 rle with old method : 0.009518146514892578 time for calcul the mask position with numpy : 0.040731191635131836 nb_pixel_total : 28730 time to create 1 rle with old method : 0.0331573486328125 time for calcul the mask position with numpy : 0.037073373794555664 nb_pixel_total : 53375 time to create 1 rle with old method : 0.06784915924072266 time for calcul the mask position with numpy : 0.035265445709228516 nb_pixel_total : 20296 time to create 1 rle with old method : 0.022912025451660156 time for calcul the mask position with numpy : 0.03372645378112793 nb_pixel_total : 144009 time to create 1 rle with old method : 0.16256141662597656 time for calcul the mask position with numpy : 0.026302099227905273 nb_pixel_total : 33607 time to create 1 rle with old method : 0.0389556884765625 time for calcul the mask position with numpy : 0.02519965171813965 nb_pixel_total : 71590 time to create 1 rle with old method : 0.0799558162689209 time for calcul the mask position with numpy : 0.025461435317993164 nb_pixel_total : 38013 time to create 1 rle with old method : 0.043294668197631836 create new chi : 2.6481714248657227 time to delete rle : 0.0017819404602050781 batch 1 Loaded 33 chid ids of type : 3594 ++++++++++++++++++++++++++++Number RLEs to save : 10665 TO DO : save crop sub photo not yet done ! save time : 0.7316319942474365 nb_obj : 11 nb_hashtags : 4 time to prepare the origin masks : 10.191468477249146 time for calcul the mask position with numpy : 0.32630157470703125 nb_pixel_total : 4070822 time to create 1 rle with new method : 1.1069574356079102 time for calcul the mask position with numpy : 0.0321199893951416 nb_pixel_total : 20405 time to create 1 rle with old method : 0.024341344833374023 time for calcul the mask position with numpy : 0.027805805206298828 nb_pixel_total : 21277 time to create 1 rle with old method : 0.024324893951416016 time for calcul the mask position with numpy : 0.02497386932373047 nb_pixel_total : 10591 time to create 1 rle with old method : 0.012146949768066406 time for calcul the mask position with numpy : 0.025249719619750977 nb_pixel_total : 8759 time to create 1 rle with old method : 0.010077953338623047 time for calcul the mask position with numpy : 0.32164573669433594 nb_pixel_total : 4036845 time to create 1 rle with new method : 1.5335605144500732 time for calcul the mask position with numpy : 0.031188011169433594 nb_pixel_total : 38223 time to create 1 rle with old method : 0.042948246002197266 time for calcul the mask position with numpy : 0.028221607208251953 nb_pixel_total : 11761 time to create 1 rle with old method : 0.0130462646484375 time for calcul the mask position with numpy : 0.025646686553955078 nb_pixel_total : 13014 time to create 1 rle with old method : 0.014544248580932617 time for calcul the mask position with numpy : 0.025864601135253906 nb_pixel_total : 12773 time to create 1 rle with old method : 0.014484882354736328 time for calcul the mask position with numpy : 0.025938749313354492 nb_pixel_total : 9525 time to create 1 rle with old method : 0.011008739471435547 time for calcul the mask position with numpy : 0.026627779006958008 nb_pixel_total : 40405 time to create 1 rle with old method : 0.045456647872924805 create new chi : 3.855412244796753 time to delete rle : 0.0016412734985351562 batch 1 Loaded 26 chid ids of type : 3594 +++++++++++++++Number RLEs to save : 9462 TO DO : save crop sub photo not yet done ! save time : 0.6407725811004639 map_output_result : {1359350759: (0.0, 'Should be the crop_list due to order', 0), 1359350600: (0.0, 'Should be the crop_list due to order', 0), 1359350584: (0.0, 'Should be the crop_list due to order', 0), 1359350569: (0.0, 'Should be the crop_list due to order', 0), 1359350550: (0.0, 'Should be the crop_list due to order', 0), 1359350520: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1359350759, 1359350600, 1359350584, 1359350569, 1359350550, 1359350520] Looping around the photos to save general results len do output : 6 /1359350759.Didn't retrieve data . /1359350600.Didn't retrieve data . /1359350584.Didn't retrieve data . /1359350569.Didn't retrieve data . /1359350550.Didn't retrieve data . /1359350520.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350759', None, None, None, None, None, '2893741') ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350600', None, None, None, None, None, '2893741') ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350584', None, None, None, None, None, '2893741') ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350569', None, None, None, None, None, '2893741') ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350550', None, None, None, None, None, '2893741') ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350520', None, None, None, None, None, '2893741') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 18 time used for this insertion : 0.012542009353637695 save_final save missing photos in datou_result : time spend for datou_step_exec : 89.40512108802795 time spend to save output : 0.012920379638671875 total time spend for step 3 : 89.41804146766663 step4:ventilate_hashtags_in_portfolio Mon May 19 14:44:26 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 23118015 get user id for portfolio 23118015 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23118015 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_fonce','environnement','metal','flou','mal_croppe','pehd','pet_clair','autre','papier','background','carton')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23118015 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_fonce','environnement','metal','flou','mal_croppe','pehd','pet_clair','autre','papier','background','carton')) AND mptpi.`min_score`=0.5 To do To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23118015 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_fonce','environnement','metal','flou','mal_croppe','pehd','pet_clair','autre','papier','background','carton')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/23118579,23118580,23118581,23118582,23118583,23118584,23118585,23118586,23118587,23118588,23118589?tags=pet_fonce,environnement,metal,flou,mal_croppe,pehd,pet_clair,autre,papier,background,carton Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1359350759, 1359350600, 1359350584, 1359350569, 1359350550, 1359350520] Looping around the photos to save general results len do output : 1 /23118015. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350759', None, None, None, None, None, '2893741') ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350600', None, None, None, None, None, '2893741') ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350584', None, None, None, None, None, '2893741') ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350569', None, None, None, None, None, '2893741') ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350550', None, None, None, None, None, '2893741') ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350520', None, None, None, None, None, '2893741') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 7 time used for this insertion : 0.014954566955566406 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.8845431804656982 time spend to save output : 0.015334844589233398 total time spend for step 4 : 0.8998780250549316 step5:final Mon May 19 14:44:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1359350759: ('0.18118987911522633',), 1359350600: ('0.18118987911522633',), 1359350584: ('0.18118987911522633',), 1359350569: ('0.18118987911522633',), 1359350550: ('0.18118987911522633',), 1359350520: ('0.18118987911522633',)} new output for save of step final : {1359350759: ('0.18118987911522633',), 1359350600: ('0.18118987911522633',), 1359350584: ('0.18118987911522633',), 1359350569: ('0.18118987911522633',), 1359350550: ('0.18118987911522633',), 1359350520: ('0.18118987911522633',)} [1359350759, 1359350600, 1359350584, 1359350569, 1359350550, 1359350520] Looping around the photos to save general results len do output : 6 /1359350759.Didn't retrieve data . /1359350600.Didn't retrieve data . /1359350584.Didn't retrieve data . /1359350569.Didn't retrieve data . /1359350550.Didn't retrieve data . /1359350520.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350759', None, None, None, None, None, '2893741') ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350600', None, None, None, None, None, '2893741') ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350584', None, None, None, None, None, '2893741') ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350569', None, None, None, None, None, '2893741') ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350550', None, None, None, None, None, '2893741') ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350520', None, None, None, None, None, '2893741') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 18 time used for this insertion : 0.013452291488647461 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.12279558181762695 time spend to save output : 0.013839006423950195 total time spend for step 5 : 0.13663458824157715 step6:blur_detection Mon May 19 14:44:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1747658427_2111745_1359350759_82616e147c572a52d750dd1b0afdc508.jpg resize: (2160, 3840) 1359350759 -5.806780816800042 treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0.jpg resize: (2160, 3840) 1359350600 -5.843652388364947 treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7.jpg resize: (2160, 3840) 1359350584 -5.634616886161984 treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f.jpg resize: (2160, 3840) 1359350569 -5.910552255275841 treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf.jpg resize: (2160, 3840) 1359350550 -5.540101977732239 treat image : temp/1747658427_2111745_1359350520_b9ca1a360f1c8bfc100b721e3e0b11b7.jpg resize: (2160, 3840) 1359350520 -3.34630276558868 treat image : temp/1747658427_2111745_1359350759_82616e147c572a52d750dd1b0afdc508_rle_crop_3803833005_0.png resize: (645, 255) 1359377296 -2.949633500566977 treat image : temp/1747658427_2111745_1359350759_82616e147c572a52d750dd1b0afdc508_rle_crop_3803833013_0.png resize: (263, 276) 1359377297 -3.118058243120917 treat image : temp/1747658427_2111745_1359350759_82616e147c572a52d750dd1b0afdc508_rle_crop_3803833004_0.png resize: (613, 525) 1359377298 -3.9573705235584247 treat image : temp/1747658427_2111745_1359350759_82616e147c572a52d750dd1b0afdc508_rle_crop_3803833009_0.png resize: (223, 361) 1359377299 -3.8661859969233134 treat image : temp/1747658427_2111745_1359350759_82616e147c572a52d750dd1b0afdc508_rle_crop_3803833006_0.png resize: (465, 830) 1359377300 -2.255468172785546 treat image : temp/1747658427_2111745_1359350759_82616e147c572a52d750dd1b0afdc508_rle_crop_3803833010_0.png resize: (165, 164) 1359377301 -3.4308705472137744 treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833029_0.png resize: (205, 197) 1359377302 -2.528601757769309 treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833019_0.png resize: (380, 172) 1359377303 -1.9938174911489999 treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833031_0.png resize: (143, 171) 1359377304 -2.861310270991211 treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833014_0.png resize: (382, 688) 1359377305 -2.966831026242682 treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833030_0.png resize: (130, 167) 1359377306 -2.144279126101161 treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833027_0.png resize: (66, 150) 1359377307 -4.089054368942249 treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833015_0.png resize: (117, 113) 1359377308 -1.5652498023438246 treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833026_0.png resize: (754, 887) 1359377309 -3.01499711306427 treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833023_0.png resize: (324, 712) 1359377310 -3.8519063148685757 treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833028_0.png resize: (241, 387) 1359377311 -1.4345717170470245 treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833016_0.png resize: (258, 116) 1359377312 -3.2095205030785823 treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833022_0.png resize: (154, 129) 1359377313 -1.5761781759826248 treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833018_0.png resize: (101, 149) 1359377314 -3.1360402646796155 treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833032_0.png resize: (586, 536) 1359377315 -3.2694639683987656 treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833021_0.png resize: (133, 272) 1359377316 -2.735465734588367 treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833034_0.png resize: (333, 262) 1359377317 -0.550750718272572 treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833017_0.png resize: (413, 542) 1359377318 -4.530232889736191 treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833025_0.png resize: (189, 343) 1359377319 -3.0790880531172595 treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833033_0.png resize: (211, 122) 1359377320 -4.67247198241418 treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833020_0.png resize: (285, 148) 1359377321 -4.045311993254064 treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833036_0.png resize: (177, 197) 1359377322 -2.001913283414083 treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833041_0.png resize: (263, 283) 1359377323 -3.316641649634279 treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833043_0.png resize: (202, 267) 1359377324 -0.8973398801466823 treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833037_0.png resize: (257, 428) 1359377325 -2.940360404142142 treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833046_0.png resize: (134, 122) 1359377326 -4.245063004149646 treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833040_0.png resize: (131, 112) 1359377327 -3.2026235431817582 treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833045_0.png resize: (101, 142) 1359377328 -1.1828447947785552 treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833038_0.png resize: (128, 274) 1359377329 -3.226610392423624 treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833050_0.png resize: (280, 202) 1359377330 -3.2488348184954425 treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833051_0.png resize: (370, 155) 1359377331 -4.111624661114067 treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833070_0.png resize: (338, 200) 1359377332 -3.3934239907930266 treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833058_0.png resize: (365, 473) 1359377334 -3.8337049998154833 treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833059_0.png resize: (232, 238) 1359377335 -1.104496669171121 treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833054_0.png resize: (306, 165) 1359377336 -3.1189234501619008 treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833073_0.png resize: (467, 310) 1359377338 -2.9580901386650136 treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833052_0.png resize: (123, 189) 1359377339 -1.4084536865249047 treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833077_0.png resize: (275, 284) 1359377340 -3.470474340966866 treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833061_0.png resize: (183, 53) 1359377343 -0.8171452096985979 treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833069_0.png resize: (280, 167) 1359377344 -3.1309484828879093 treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833057_0.png resize: (265, 171) 1359377345 -2.612424692523549 treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833074_0.png resize: (250, 191) 1359377346 -1.8822400042902323 treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833068_0.png resize: (433, 783) 1359377348 -4.550283668277687 treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833075_0.png resize: (168, 182) 1359377349 -3.6677983382218824 treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833062_0.png resize: (104, 146) 1359377350 -0.5491990815900508 treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833067_0.png resize: (95, 222) 1359377352 -4.004891338653133 treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833072_0.png resize: (390, 355) 1359377353 -2.22007496617024 treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833078_0.png resize: (494, 298) 1359377354 -2.8174112835369693 treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833053_0.png resize: (361, 241) 1359377356 -1.2451379198231034 treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833076_0.png resize: (230, 156) 1359377357 -3.020501145647826 treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833060_0.png resize: (420, 361) 1359377358 -2.807737049250222 treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833065_0.png resize: (210, 318) 1359377360 -2.1055061342413204 treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833063_0.png resize: (328, 412) 1359377361 -2.013171681075626 treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833064_0.png resize: (203, 182) 1359377362 -3.5061061953370447 treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833055_0.png resize: (183, 157) 1359377363 -2.3778921850958668 treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833056_0.png resize: (504, 377) 1359377365 -4.311762475586818 treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833085_0.png resize: (142, 440) 1359377366 -2.952218535057529 treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833081_0.png resize: (226, 237) 1359377367 -1.228011788426631 treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833083_0.png resize: (186, 152) 1359377369 -3.563324075051896 treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833080_0.png resize: (358, 348) 1359377370 -2.6921353164478994 treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833089_0.png resize: (478, 633) 1359377371 -3.0227566270271287 treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833088_0.png resize: (194, 320) 1359377373 -4.2012493818735965 treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833084_0.png resize: (309, 301) 1359377374 -3.540203485254116 treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833090_0.png resize: (328, 225) 1359377375 -3.76943667465211 treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833093_0.png resize: (301, 322) 1359377377 -3.806421437535373 treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833092_0.png resize: (94, 102) 1359377378 1.3524389145218965 treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833094_0.png resize: (251, 133) 1359377379 -3.5820350050378416 treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833086_0.png resize: (94, 113) 1359377381 0.6058149504879169 treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833079_0.png resize: (238, 262) 1359377382 -3.085597465344348 treat image : temp/1747658427_2111745_1359350520_b9ca1a360f1c8bfc100b721e3e0b11b7_rle_crop_3803833096_0.png resize: (96, 133) 1359377383 -2.2217877424917387 treat image : temp/1747658427_2111745_1359350520_b9ca1a360f1c8bfc100b721e3e0b11b7_rle_crop_3803833105_0.png resize: (244, 144) 1359377384 -3.810286475653321 treat image : temp/1747658427_2111745_1359350520_b9ca1a360f1c8bfc100b721e3e0b11b7_rle_crop_3803833095_0.png resize: (285, 238) 1359377386 -3.216692851043673 treat image : temp/1747658427_2111745_1359350520_b9ca1a360f1c8bfc100b721e3e0b11b7_rle_crop_3803833097_0.png resize: (116, 132) 1359377387 -1.7045068282469904 treat image : temp/1747658427_2111745_1359350520_b9ca1a360f1c8bfc100b721e3e0b11b7_rle_crop_3803833098_0.png resize: (152, 154) 1359377388 -3.3018001600749978 treat image : temp/1747658427_2111745_1359350520_b9ca1a360f1c8bfc100b721e3e0b11b7_rle_crop_3803833102_0.png resize: (194, 200) 1359377389 -3.835350132241328 treat image : temp/1747658427_2111745_1359350520_b9ca1a360f1c8bfc100b721e3e0b11b7_rle_crop_3803833099_0.png resize: (126, 121) 1359377390 -3.6865816002202485 treat image : temp/1747658427_2111745_1359350759_82616e147c572a52d750dd1b0afdc508_rle_crop_3803833011_0.png resize: (155, 141) 1359377402 -1.9546970244859898 treat image : temp/1747658427_2111745_1359350759_82616e147c572a52d750dd1b0afdc508_rle_crop_3803833012_0.png resize: (502, 507) 1359377403 -3.0277937571358096 treat image : temp/1747658427_2111745_1359350759_82616e147c572a52d750dd1b0afdc508_rle_crop_3803833007_0.png resize: (198, 197) 1359377405 -1.5908341954575587 treat image : temp/1747658427_2111745_1359350759_82616e147c572a52d750dd1b0afdc508_rle_crop_3803833008_0.png resize: (453, 354) 1359377406 -2.515999479222492 treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833024_0.png resize: (178, 150) 1359377408 -2.92744712397144 treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833044_0.png resize: (250, 206) 1359377410 -3.6468727102935112 treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833048_0.png resize: (163, 198) 1359377411 -3.550749029409405 treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833049_0.png resize: (250, 136) 1359377412 -3.71437759953065 treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833039_0.png resize: (252, 152) 1359377414 -2.179085266143395 treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833035_0.png resize: (217, 312) 1359377415 -2.3407209199602907 treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833066_0.png resize: (262, 300) 1359377417 -3.3763025315567523 treat image : temp/1747658427_2111745_1359350520_b9ca1a360f1c8bfc100b721e3e0b11b7_rle_crop_3803833108_0.png resize: (106, 97) 1359377418 -3.5523734148542783 treat image : temp/1747658427_2111745_1359350520_b9ca1a360f1c8bfc100b721e3e0b11b7_rle_crop_3803833100_0.png resize: (217, 302) 1359377420 -2.410560985910606 treat image : temp/1747658427_2111745_1359350520_b9ca1a360f1c8bfc100b721e3e0b11b7_rle_crop_3803833103_0.png resize: (173, 275) 1359377421 -2.3810981160874705 treat image : temp/1747658427_2111745_1359350520_b9ca1a360f1c8bfc100b721e3e0b11b7_rle_crop_3803833106_0.png resize: (89, 275) 1359377422 -2.9817629215070696 treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833071_0.png resize: (172, 122) 1359377441 -4.346405287097142 treat image : temp/1747658427_2111745_1359350520_b9ca1a360f1c8bfc100b721e3e0b11b7_rle_crop_3803833104_0.png resize: (140, 138) 1359377442 -4.176482399162502 treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833047_0.png resize: (61, 248) 1359377498 -1.634210113097346 treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833042_0.png resize: (99, 151) 1359377499 -3.7783815046658438 treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833082_0.png resize: (304, 699) 1359377500 -3.825976136309616 treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833091_0.png resize: (69, 189) 1359377501 -3.9761289493499805 treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833087_0.png resize: (378, 242) 1359377502 -4.353923654178414 treat image : temp/1747658427_2111745_1359350520_b9ca1a360f1c8bfc100b721e3e0b11b7_rle_crop_3803833101_0.png resize: (1992, 2654) 1359377503 -3.1035608532704306 treat image : temp/1747658427_2111745_1359350520_b9ca1a360f1c8bfc100b721e3e0b11b7_rle_crop_3803833107_0.png resize: (242, 731) 1359377504 -3.3702472360320552 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 111 time used for this insertion : 0.03684353828430176 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 111 time used for this insertion : 0.036533355712890625 save missing photos in datou_result : time spend for datou_step_exec : 29.06043553352356 time spend to save output : 0.07875800132751465 total time spend for step 6 : 29.139193534851074 step7:brightness Mon May 19 14:44:56 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1747658427_2111745_1359350759_82616e147c572a52d750dd1b0afdc508.jpg treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0.jpg treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7.jpg treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f.jpg treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf.jpg treat image : temp/1747658427_2111745_1359350520_b9ca1a360f1c8bfc100b721e3e0b11b7.jpg treat image : temp/1747658427_2111745_1359350759_82616e147c572a52d750dd1b0afdc508_rle_crop_3803833005_0.png treat image : temp/1747658427_2111745_1359350759_82616e147c572a52d750dd1b0afdc508_rle_crop_3803833013_0.png treat image : temp/1747658427_2111745_1359350759_82616e147c572a52d750dd1b0afdc508_rle_crop_3803833004_0.png treat image : temp/1747658427_2111745_1359350759_82616e147c572a52d750dd1b0afdc508_rle_crop_3803833009_0.png treat image : temp/1747658427_2111745_1359350759_82616e147c572a52d750dd1b0afdc508_rle_crop_3803833006_0.png treat image : temp/1747658427_2111745_1359350759_82616e147c572a52d750dd1b0afdc508_rle_crop_3803833010_0.png treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833029_0.png treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833019_0.png treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833031_0.png treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833014_0.png treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833030_0.png treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833027_0.png treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833015_0.png treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833026_0.png treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833023_0.png treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833028_0.png treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833016_0.png treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833022_0.png treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833018_0.png treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833032_0.png treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833021_0.png treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833034_0.png treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833017_0.png treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833025_0.png treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833033_0.png treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833020_0.png treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833036_0.png treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833041_0.png treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833043_0.png treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833037_0.png treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833046_0.png treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833040_0.png treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833045_0.png treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833038_0.png treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833050_0.png treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833051_0.png treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833070_0.png treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833058_0.png treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833059_0.png treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833054_0.png treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833073_0.png treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833052_0.png treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833077_0.png treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833061_0.png treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833069_0.png treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833057_0.png treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833074_0.png treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833068_0.png treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833075_0.png treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833062_0.png treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833067_0.png treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833072_0.png treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833078_0.png treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833053_0.png treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833076_0.png treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833060_0.png treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833065_0.png treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833063_0.png treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833064_0.png treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833055_0.png treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833056_0.png treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833085_0.png treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833081_0.png treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833083_0.png treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833080_0.png treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833089_0.png treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833088_0.png treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833084_0.png treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833090_0.png treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833093_0.png treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833092_0.png treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833094_0.png treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833086_0.png treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833079_0.png treat image : temp/1747658427_2111745_1359350520_b9ca1a360f1c8bfc100b721e3e0b11b7_rle_crop_3803833096_0.png treat image : temp/1747658427_2111745_1359350520_b9ca1a360f1c8bfc100b721e3e0b11b7_rle_crop_3803833105_0.png treat image : temp/1747658427_2111745_1359350520_b9ca1a360f1c8bfc100b721e3e0b11b7_rle_crop_3803833095_0.png treat image : temp/1747658427_2111745_1359350520_b9ca1a360f1c8bfc100b721e3e0b11b7_rle_crop_3803833097_0.png treat image : temp/1747658427_2111745_1359350520_b9ca1a360f1c8bfc100b721e3e0b11b7_rle_crop_3803833098_0.png treat image : temp/1747658427_2111745_1359350520_b9ca1a360f1c8bfc100b721e3e0b11b7_rle_crop_3803833102_0.png treat image : temp/1747658427_2111745_1359350520_b9ca1a360f1c8bfc100b721e3e0b11b7_rle_crop_3803833099_0.png treat image : temp/1747658427_2111745_1359350759_82616e147c572a52d750dd1b0afdc508_rle_crop_3803833011_0.png treat image : temp/1747658427_2111745_1359350759_82616e147c572a52d750dd1b0afdc508_rle_crop_3803833012_0.png treat image : temp/1747658427_2111745_1359350759_82616e147c572a52d750dd1b0afdc508_rle_crop_3803833007_0.png treat image : temp/1747658427_2111745_1359350759_82616e147c572a52d750dd1b0afdc508_rle_crop_3803833008_0.png treat image : temp/1747658427_2111745_1359350600_1c3d52b821b70654cb950fe63d6d29e0_rle_crop_3803833024_0.png treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833044_0.png treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833048_0.png treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833049_0.png treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833039_0.png treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833035_0.png treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833066_0.png treat image : temp/1747658427_2111745_1359350520_b9ca1a360f1c8bfc100b721e3e0b11b7_rle_crop_3803833108_0.png treat image : temp/1747658427_2111745_1359350520_b9ca1a360f1c8bfc100b721e3e0b11b7_rle_crop_3803833100_0.png treat image : temp/1747658427_2111745_1359350520_b9ca1a360f1c8bfc100b721e3e0b11b7_rle_crop_3803833103_0.png treat image : temp/1747658427_2111745_1359350520_b9ca1a360f1c8bfc100b721e3e0b11b7_rle_crop_3803833106_0.png treat image : temp/1747658427_2111745_1359350569_82af967d4e67e2e52882b7c22002dd5f_rle_crop_3803833071_0.png treat image : temp/1747658427_2111745_1359350520_b9ca1a360f1c8bfc100b721e3e0b11b7_rle_crop_3803833104_0.png treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833047_0.png treat image : temp/1747658427_2111745_1359350584_3b6c4eb6b07e0e537212eb75df8436b7_rle_crop_3803833042_0.png treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833082_0.png treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833091_0.png treat image : temp/1747658427_2111745_1359350550_e496b6c38c403ff823aabe4799f283cf_rle_crop_3803833087_0.png treat image : temp/1747658427_2111745_1359350520_b9ca1a360f1c8bfc100b721e3e0b11b7_rle_crop_3803833101_0.png treat image : temp/1747658427_2111745_1359350520_b9ca1a360f1c8bfc100b721e3e0b11b7_rle_crop_3803833107_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 111 time used for this insertion : 0.022144079208374023 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 111 time used for this insertion : 0.03870844841003418 save missing photos in datou_result : time spend for datou_step_exec : 7.1861732006073 time spend to save output : 0.06553030014038086 total time spend for step 7 : 7.251703500747681 step8:velours_tree Mon May 19 14:45:03 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.09766936302185059 time spend to save output : 4.3392181396484375e-05 total time spend for step 8 : 0.09771275520324707 step9:send_mail_cod Mon May 19 14:45:03 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P23118015_19-05-2025_14_45_03.pdf 23118579 imagette231185791747658703 23118581 change filename to text .change filename to text .imagette231185811747658703 23118582 imagette231185821747658703 23118583 imagette231185831747658703 23118584 imagette231185841747658703 23118585 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette231185851747658703 23118586 imagette231185861747658704 23118587 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette231185871747658704 23118588 imagette231185881747658707 23118589 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette231185891747658707 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=23118015 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/23118579,23118580,23118581,23118582,23118583,23118584,23118585,23118586,23118587,23118588,23118589?tags=pet_fonce,environnement,metal,flou,mal_croppe,pehd,pet_clair,autre,papier,background,carton args[1359350759] : ((1359350759, -5.806780816800042, 492609224), (1359350759, 1.3047693634235467, 2107752395), '0.18118987911522633') We are sending mail with results at report@fotonower.com args[1359350600] : ((1359350600, -5.843652388364947, 492609224), (1359350600, 1.1830139397498165, 2107752395), '0.18118987911522633') We are sending mail with results at report@fotonower.com args[1359350584] : ((1359350584, -5.634616886161984, 492609224), (1359350584, 0.5966065203407857, 2107752395), '0.18118987911522633') We are sending mail with results at report@fotonower.com args[1359350569] : ((1359350569, -5.910552255275841, 492609224), (1359350569, 0.8866254333328292, 2107752395), '0.18118987911522633') We are sending mail with results at report@fotonower.com args[1359350550] : ((1359350550, -5.540101977732239, 492609224), (1359350550, 0.29672108728798463, 2107752395), '0.18118987911522633') We are sending mail with results at report@fotonower.com args[1359350520] : ((1359350520, -3.34630276558868, 492609224), (1359350520, -0.32503618633316333, 496442774), '0.18118987911522633') We are sending mail with results at report@fotonower.com refus_total : 0.18118987911522633 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=23118015 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23118015_19-05-2025_14_45_03.pdf results_Auto_P23118015_19-05-2025_14_45_03.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23118015_19-05-2025_14_45_03.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','23118015','results_Auto_P23118015_19-05-2025_14_45_03.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23118015_19-05-2025_14_45_03.pdf','pdf','','0.93','0.18118987911522633') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/23118015

https://www.fotonower.com/image?json=false&list_photos_id=1359350759
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359350600
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359350584
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359350569
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359350550
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359350520
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 18.12%
Veuillez trouver les photos des contaminants.

exemples de contaminants: metal: https://www.fotonower.com/view/23118581?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/23118585?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/23118587?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/23118589?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23118015_19-05-2025_14_45_03.pdf.

Lien vers velours :https://www.fotonower.com/velours/23118579,23118580,23118581,23118582,23118583,23118584,23118585,23118586,23118587,23118588,23118589?tags=pet_fonce,environnement,metal,flou,mal_croppe,pehd,pet_clair,autre,papier,background,carton.


L'équipe Fotonower 202 b'' Server: nginx Date: Mon, 19 May 2025 12:45:10 GMT Content-Length: 0 Connection: close X-Message-Id: 8-T1Xc4lTKWKwk1GKvPtcg Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1359350759, 1359350600, 1359350584, 1359350569, 1359350550, 1359350520] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350759', None, None, None, None, None, '2893741') ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350600', None, None, None, None, None, '2893741') ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350584', None, None, None, None, None, '2893741') ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350569', None, None, None, None, None, '2893741') ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350550', None, None, None, None, None, '2893741') ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350520', None, None, None, None, None, '2893741') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.09114837646484375 save_final save missing photos in datou_result : time spend for datou_step_exec : 7.141446828842163 time spend to save output : 0.09132671356201172 total time spend for step 9 : 7.232773542404175 step10:split_time_score Mon May 19 14:45:10 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('12', 6),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 19052025 23118015 Nombre de photos uploadées : 6 / 23040 (0%) 19052025 23118015 Nombre de photos taguées (types de déchets): 0 / 6 (0%) 19052025 23118015 Nombre de photos taguées (volume) : 0 / 6 (0%) elapsed_time : load_data_split_time_score 3.0994415283203125e-06 elapsed_time : order_list_meta_photo_and_scores 1.0013580322265625e-05 ?????? elapsed_time : fill_and_build_computed_from_old_data 0.0006933212280273438 elapsed_time : insert_dashboard_record_day_entry 0.03297114372253418 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.12806391095562103 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23104924_19-05-2025_10_08_08.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23104924 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23104924 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23104935 order by id desc limit 1 Qualite : 0.11080586682800225 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23106802_19-05-2025_11_02_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23106802 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23106802 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23110969 order by id desc limit 1 Qualite : 0.10507966553069274 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23110971_19-05-2025_12_21_54.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23110971 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23110971 AND mptpi.`type`=3594 To do Qualite : 0.18118987911522633 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23118015_19-05-2025_14_45_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23118015 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23118015 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23118018 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23118039 order by id desc limit 1 NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'19052025': {'nb_upload': 6, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1359350759, 1359350600, 1359350584, 1359350569, 1359350550, 1359350520] Looping around the photos to save general results len do output : 1 /23118015Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350759', None, None, None, None, None, '2893741') ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350600', None, None, None, None, None, '2893741') ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350584', None, None, None, None, None, '2893741') ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350569', None, None, None, None, None, '2893741') ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350550', None, None, None, None, None, '2893741') ('3318', None, None, None, None, None, None, None, '2893741') ('3318', '23118015', '1359350520', None, None, None, None, None, '2893741') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 7 time used for this insertion : 0.05505633354187012 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.390651702880859 time spend to save output : 0.05533170700073242 total time spend for step 10 : 4.445983409881592 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 6 set_done_treatment 126.50user 119.08system 4:50.36elapsed 84%CPU (0avgtext+0avgdata 4223792maxresident)k 625640inputs+145280outputs (9088major+10658893minor)pagefaults 0swaps