python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 11332 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['2801881'] with mtr_portfolio_ids : ['22758212'] and first list_photo_ids : [] new path : /proc/11332/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 8 ; length of list_pids : 8 ; length of list_args : 8 time to download the photos : 1.1723601818084717 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Wed May 7 09:10:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10814 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-05-07 09:10:31.582952: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-05-07 09:10:31.607280: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-05-07 09:10:31.609423: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f6dd4000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-05-07 09:10:31.609486: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-05-07 09:10:31.613341: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-05-07 09:10:31.858832: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x21e353e0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-05-07 09:10:31.858877: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-05-07 09:10:31.860285: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-07 09:10:31.860669: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-07 09:10:31.863873: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-07 09:10:31.866812: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-07 09:10:31.867346: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-07 09:10:31.872145: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-07 09:10:31.873453: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-07 09:10:31.878880: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-07 09:10:31.880751: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-07 09:10:31.880835: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-07 09:10:31.881777: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-07 09:10:31.881797: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-07 09:10:31.881809: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-07 09:10:31.883539: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10023 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-05-07 09:10:32.161047: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-07 09:10:32.161175: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-07 09:10:32.161202: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-07 09:10:32.161227: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-07 09:10:32.161252: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-07 09:10:32.161276: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-07 09:10:32.161299: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-07 09:10:32.161324: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-07 09:10:32.163472: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-07 09:10:32.164677: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-07 09:10:32.164705: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-07 09:10:32.164718: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-07 09:10:32.164730: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-07 09:10:32.164742: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-07 09:10:32.164754: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-07 09:10:32.164766: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-07 09:10:32.164778: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-07 09:10:32.165933: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-07 09:10:32.165965: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-07 09:10:32.165973: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-07 09:10:32.165980: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-07 09:10:32.167177: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10023 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-05-07 09:10:42.934892: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-07 09:10:43.114601: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 8 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 3.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 27 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 41 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 34 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 26.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 16 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 21.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 41 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 55.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 37 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 43 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 34.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 19 Detection mask done ! Trying to reset tf kernel 11945 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5525 tf kernel not reseted sub process len(results) : 8 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 8 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10814 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.003570556640625 nb_pixel_total : 117273 time to create 1 rle with old method : 0.20835661888122559 length of segment : 262 time for calcul the mask position with numpy : 0.00043582916259765625 nb_pixel_total : 12591 time to create 1 rle with old method : 0.015219688415527344 length of segment : 206 time for calcul the mask position with numpy : 0.0007240772247314453 nb_pixel_total : 33739 time to create 1 rle with old method : 0.04143691062927246 length of segment : 220 time for calcul the mask position with numpy : 0.0009615421295166016 nb_pixel_total : 41626 time to create 1 rle with old method : 0.04937267303466797 length of segment : 278 time for calcul the mask position with numpy : 0.00015020370483398438 nb_pixel_total : 6627 time to create 1 rle with old method : 0.007793426513671875 length of segment : 89 time for calcul the mask position with numpy : 0.0004379749298095703 nb_pixel_total : 29166 time to create 1 rle with old method : 0.033746957778930664 length of segment : 210 time for calcul the mask position with numpy : 0.0009076595306396484 nb_pixel_total : 61333 time to create 1 rle with old method : 0.07095932960510254 length of segment : 339 time for calcul the mask position with numpy : 0.00395512580871582 nb_pixel_total : 179486 time to create 1 rle with new method : 0.00926518440246582 length of segment : 402 time for calcul the mask position with numpy : 0.002735137939453125 nb_pixel_total : 181803 time to create 1 rle with new method : 0.010028362274169922 length of segment : 734 time for calcul the mask position with numpy : 0.001760721206665039 nb_pixel_total : 110156 time to create 1 rle with old method : 0.1424703598022461 length of segment : 448 time for calcul the mask position with numpy : 0.0004813671112060547 nb_pixel_total : 26351 time to create 1 rle with old method : 0.030962467193603516 length of segment : 199 time for calcul the mask position with numpy : 0.0007894039154052734 nb_pixel_total : 47406 time to create 1 rle with old method : 0.05514883995056152 length of segment : 259 time for calcul the mask position with numpy : 0.00018310546875 nb_pixel_total : 8863 time to create 1 rle with old method : 0.010683536529541016 length of segment : 131 time for calcul the mask position with numpy : 0.0023016929626464844 nb_pixel_total : 155378 time to create 1 rle with new method : 0.0074465274810791016 length of segment : 312 time for calcul the mask position with numpy : 0.0005650520324707031 nb_pixel_total : 20390 time to create 1 rle with old method : 0.02390003204345703 length of segment : 254 time for calcul the mask position with numpy : 0.0005462169647216797 nb_pixel_total : 23256 time to create 1 rle with old method : 0.028304100036621094 length of segment : 153 time for calcul the mask position with numpy : 0.0021865367889404297 nb_pixel_total : 121559 time to create 1 rle with old method : 0.14445066452026367 length of segment : 319 time for calcul the mask position with numpy : 0.0005040168762207031 nb_pixel_total : 16292 time to create 1 rle with old method : 0.02230095863342285 length of segment : 124 time for calcul the mask position with numpy : 0.00036263465881347656 nb_pixel_total : 13354 time to create 1 rle with old method : 0.0213162899017334 length of segment : 105 time for calcul the mask position with numpy : 0.0010874271392822266 nb_pixel_total : 40282 time to create 1 rle with old method : 0.05199551582336426 length of segment : 240 time for calcul the mask position with numpy : 0.0013742446899414062 nb_pixel_total : 66477 time to create 1 rle with old method : 0.07935357093811035 length of segment : 370 time for calcul the mask position with numpy : 0.0008912086486816406 nb_pixel_total : 44960 time to create 1 rle with old method : 0.05201554298400879 length of segment : 243 time for calcul the mask position with numpy : 0.001268625259399414 nb_pixel_total : 63708 time to create 1 rle with old method : 0.07394647598266602 length of segment : 256 time for calcul the mask position with numpy : 0.00022411346435546875 nb_pixel_total : 8029 time to create 1 rle with old method : 0.009145021438598633 length of segment : 132 time for calcul the mask position with numpy : 0.0005905628204345703 nb_pixel_total : 23260 time to create 1 rle with old method : 0.02702498435974121 length of segment : 169 time for calcul the mask position with numpy : 0.0005488395690917969 nb_pixel_total : 27916 time to create 1 rle with old method : 0.03145194053649902 length of segment : 235 time for calcul the mask position with numpy : 0.0008740425109863281 nb_pixel_total : 45624 time to create 1 rle with old method : 0.05785632133483887 length of segment : 330 time for calcul the mask position with numpy : 0.0008082389831542969 nb_pixel_total : 34109 time to create 1 rle with old method : 0.03953814506530762 length of segment : 241 time for calcul the mask position with numpy : 0.0008366107940673828 nb_pixel_total : 23162 time to create 1 rle with old method : 0.026526212692260742 length of segment : 216 time for calcul the mask position with numpy : 0.001317739486694336 nb_pixel_total : 71200 time to create 1 rle with old method : 0.08409333229064941 length of segment : 637 time for calcul the mask position with numpy : 0.0012526512145996094 nb_pixel_total : 47365 time to create 1 rle with old method : 0.07863807678222656 length of segment : 210 time for calcul the mask position with numpy : 0.0015752315521240234 nb_pixel_total : 23219 time to create 1 rle with old method : 0.026972293853759766 length of segment : 267 time for calcul the mask position with numpy : 0.0008113384246826172 nb_pixel_total : 26323 time to create 1 rle with old method : 0.030797243118286133 length of segment : 251 time for calcul the mask position with numpy : 0.0007348060607910156 nb_pixel_total : 30614 time to create 1 rle with old method : 0.03532552719116211 length of segment : 223 time for calcul the mask position with numpy : 0.0008459091186523438 nb_pixel_total : 28597 time to create 1 rle with old method : 0.037816762924194336 length of segment : 307 time for calcul the mask position with numpy : 0.0025424957275390625 nb_pixel_total : 53739 time to create 1 rle with old method : 0.061289072036743164 length of segment : 397 time for calcul the mask position with numpy : 0.0013875961303710938 nb_pixel_total : 24711 time to create 1 rle with old method : 0.028475284576416016 length of segment : 280 time for calcul the mask position with numpy : 0.0011518001556396484 nb_pixel_total : 29670 time to create 1 rle with old method : 0.0341796875 length of segment : 188 time for calcul the mask position with numpy : 0.0013017654418945312 nb_pixel_total : 23246 time to create 1 rle with old method : 0.02695155143737793 length of segment : 169 time for calcul the mask position with numpy : 0.001425027847290039 nb_pixel_total : 19529 time to create 1 rle with old method : 0.024033546447753906 length of segment : 183 time for calcul the mask position with numpy : 0.00116729736328125 nb_pixel_total : 23243 time to create 1 rle with old method : 0.028511762619018555 length of segment : 181 time for calcul the mask position with numpy : 0.0016019344329833984 nb_pixel_total : 39706 time to create 1 rle with old method : 0.04649043083190918 length of segment : 253 time for calcul the mask position with numpy : 0.006983757019042969 nb_pixel_total : 54972 time to create 1 rle with old method : 0.06380343437194824 length of segment : 313 time for calcul the mask position with numpy : 0.0024073123931884766 nb_pixel_total : 67636 time to create 1 rle with old method : 0.07644915580749512 length of segment : 318 time for calcul the mask position with numpy : 0.0012946128845214844 nb_pixel_total : 28360 time to create 1 rle with old method : 0.03216743469238281 length of segment : 214 time for calcul the mask position with numpy : 0.0006592273712158203 nb_pixel_total : 22735 time to create 1 rle with old method : 0.028012990951538086 length of segment : 208 time for calcul the mask position with numpy : 0.0006494522094726562 nb_pixel_total : 35066 time to create 1 rle with old method : 0.04226064682006836 length of segment : 151 time for calcul the mask position with numpy : 0.008321046829223633 nb_pixel_total : 83810 time to create 1 rle with old method : 0.09829497337341309 length of segment : 534 time for calcul the mask position with numpy : 0.003201007843017578 nb_pixel_total : 78801 time to create 1 rle with old method : 0.08927726745605469 length of segment : 341 time for calcul the mask position with numpy : 0.0006318092346191406 nb_pixel_total : 8862 time to create 1 rle with old method : 0.010702133178710938 length of segment : 164 time for calcul the mask position with numpy : 0.0036077499389648438 nb_pixel_total : 118559 time to create 1 rle with old method : 0.13525032997131348 length of segment : 505 time for calcul the mask position with numpy : 0.004271507263183594 nb_pixel_total : 126118 time to create 1 rle with old method : 0.1462547779083252 length of segment : 561 time for calcul the mask position with numpy : 0.0007460117340087891 nb_pixel_total : 19472 time to create 1 rle with old method : 0.023010969161987305 length of segment : 160 time for calcul the mask position with numpy : 0.00520014762878418 nb_pixel_total : 151423 time to create 1 rle with new method : 0.01006007194519043 length of segment : 523 time for calcul the mask position with numpy : 0.0007169246673583984 nb_pixel_total : 11633 time to create 1 rle with old method : 0.013589620590209961 length of segment : 334 time for calcul the mask position with numpy : 0.0019431114196777344 nb_pixel_total : 100839 time to create 1 rle with old method : 0.1132667064666748 length of segment : 295 time for calcul the mask position with numpy : 0.0001742839813232422 nb_pixel_total : 5225 time to create 1 rle with old method : 0.00603795051574707 length of segment : 64 time for calcul the mask position with numpy : 0.0003204345703125 nb_pixel_total : 9374 time to create 1 rle with old method : 0.010983943939208984 length of segment : 120 time for calcul the mask position with numpy : 0.0006375312805175781 nb_pixel_total : 22779 time to create 1 rle with old method : 0.026025772094726562 length of segment : 187 time for calcul the mask position with numpy : 0.00041294097900390625 nb_pixel_total : 14306 time to create 1 rle with old method : 0.01722860336303711 length of segment : 128 time for calcul the mask position with numpy : 0.00041365623474121094 nb_pixel_total : 14354 time to create 1 rle with old method : 0.017635583877563477 length of segment : 134 time for calcul the mask position with numpy : 0.006913423538208008 nb_pixel_total : 246681 time to create 1 rle with new method : 0.012383699417114258 length of segment : 824 time for calcul the mask position with numpy : 0.0038826465606689453 nb_pixel_total : 99413 time to create 1 rle with old method : 0.12125444412231445 lengt7769 time to create 1 rle with old method : 0.009249687194824219 length of segment : 141 time for calcul the mask position with numpy : 0.002293109893798828 nb_pixel_total : 105874 time to create 1 rle with old method : 0.12193036079406738 length of segment : 427 time for calcul the mask position with numpy : 0.0012660026550292969 nb_pixel_total : 60205 time to create 1 rle with old method : 0.06839537620544434 length of segment : 420 time for calcul the mask position with numpy : 0.00037217140197753906 nb_pixel_total : 9083 time to create 1 rle with old method : 0.010480403900146484 length of segment : 125 time spent for convertir_results : 12.112500190734863 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 137 chid ids of type : 3594 Number RLEs to save : 37445 save missing photos in datou_result : time spend for datou_step_exec : 66.61775875091553 time spend to save output : 18.750367164611816 total time spend for step 1 : 85.36812591552734 step2:crop_condition Wed May 7 09:11:54 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 8 ! batch 1 Loaded 137 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 71 About to insert : list_path_to_insert length 71 new photo from crops ! About to upload 71 photos upload in portfolio : 3736932 init cache_photo without model_param we have 71 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746601929_11332 we have uploaded 71 photos in the portfolio 3736932 time of upload the photos Elapsed time : 18.822795629501343 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 14 About to insert : list_path_to_insert length 14 new photo from crops ! About to upload 14 photos upload in portfolio : 3736932 init cache_photo without model_param we have 14 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746601951_11332 we have uploaded 14 photos in the portfolio 3736932 time of upload the photos Elapsed time : 3.50956654548645 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 47 About to insert : list_path_to_insert length 47 new photo from crops ! About to upload 47 photos upload in portfolio : 3736932 init cache_photo without model_param we have 47 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746601974_11332 we have uploaded 47 photos in the portfolio 3736932 time of upload the photos Elapsed time : 14.550227165222168 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 3736932 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746601990_11332 we have uploaded 5 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.8627521991729736 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1356625852, 1356625849, 1356625846, 1356625842, 1356625822, 1356625767, 1356625765, 1356625753] Looping around the photos to save general results len do output : 137 /1356630543Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630545Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630547Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630549Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630550Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630552Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630553Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630555Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630557Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630558Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630560Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630562Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630563Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630565Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630567Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630568Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630570Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630572Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630574Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630575Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630577Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630579Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630580Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630582Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630584Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630586Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630587Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630589Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630591Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630592Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630594Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630596Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630597Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630600Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630601Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630602Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630605Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630606Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630608Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630610Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630611Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630614Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630615Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630616Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630618Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630619Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630620Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630622Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630623Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630624Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630626Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630627Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630628Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630630Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630631Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630632Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630634Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630635Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630637Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630638Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630639Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630641Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630642Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630643Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630645Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630646Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630648Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630649Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630650Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630652Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630653Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630704Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630706Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630707Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630708Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630711Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630713Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630714Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630716Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630717Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630719Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630721Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630722Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630724Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630726Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630853Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630854Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630856Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630857Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630858Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630859Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630860Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630861Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630862Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630863Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630864Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630865Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630866Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630867Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630868Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630869Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630870Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630871Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630872Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630873Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630874Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630875Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630876Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630877Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630878Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630879Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630880Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630881Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630882Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630883Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630884Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630885Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630886Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630887Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630888Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630889Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630890Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630891Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630892Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630893Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630894Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630895Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630896Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630897Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630898Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630899Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630900Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630902Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630903Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630904Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630905Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356630906Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625852', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625849', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625846', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625842', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625822', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625767', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625765', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625753', None, None, None, None, None, '2801881') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 419 time used for this insertion : 2.4362988471984863 save_final save missing photos in datou_result : time spend for datou_step_exec : 77.60275864601135 time spend to save output : 2.4400620460510254 total time spend for step 2 : 80.04282069206238 step3:rle_unique_nms_with_priority Wed May 7 09:13:14 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 137 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 17 nb_hashtags : 3 time to prepare the origin masks : 5.287376165390015 time for calcul the mask position with numpy : 0.4373319149017334 nb_pixel_total : 5873239 time to create 1 rle with new method : 0.8506731986999512 time for calcul the mask position with numpy : 0.021350383758544922 nb_pixel_total : 121559 time to create 1 rle with old method : 0.13483285903930664 time for calcul the mask position with numpy : 0.020030498504638672 nb_pixel_total : 23256 time to create 1 rle with old method : 0.025992631912231445 time for calcul the mask position with numpy : 0.01974201202392578 nb_pixel_total : 20390 time to create 1 rle with old method : 0.021829843521118164 time for calcul the mask position with numpy : 0.020866870880126953 nb_pixel_total : 155378 time to create 1 rle with new method : 0.5580761432647705 time for calcul the mask position with numpy : 0.02135181427001953 nb_pixel_total : 8863 time to create 1 rle with old method : 0.009636402130126953 time for calcul the mask position with numpy : 0.020473480224609375 nb_pixel_total : 47406 time to create 1 rle with old method : 0.052520036697387695 time for calcul the mask position with numpy : 0.020607471466064453 nb_pixel_total : 26351 time to create 1 rle with old method : 0.02961897850036621 time for calcul the mask position with numpy : 0.02097344398498535 nb_pixel_total : 110156 time to create 1 rle with old method : 0.1235654354095459 time for calcul the mask position with numpy : 0.021984100341796875 nb_pixel_total : 181803 time to create 1 rle with new method : 0.5882725715637207 time for calcul the mask position with numpy : 0.024773359298706055 nb_pixel_total : 179486 time to create 1 rle with new method : 0.7699570655822754 time for calcul the mask position with numpy : 0.021893024444580078 nb_pixel_total : 61333 time to create 1 rle with old method : 0.0917520523071289 time for calcul the mask position with numpy : 0.023389101028442383 nb_pixel_total : 29166 time to create 1 rle with old method : 0.041474103927612305 time for calcul the mask position with numpy : 0.023305177688598633 nb_pixel_total : 6627 time to create 1 rle with old method : 0.011043548583984375 time for calcul the mask position with numpy : 0.02362823486328125 nb_pixel_total : 41624 time to create 1 rle with old method : 0.04854536056518555 time for calcul the mask position with numpy : 0.024008750915527344 nb_pixel_total : 33739 time to create 1 rle with old method : 0.03784608840942383 time for calcul the mask position with numpy : 0.021451950073242188 nb_pixel_total : 12591 time to create 1 rle with old method : 0.015126705169677734 time for calcul the mask position with numpy : 0.027053117752075195 nb_pixel_total : 117273 time to create 1 rle with old method : 0.13999629020690918 create new chi : 4.456430673599243 time to delete rle : 0.026863574981689453 batch 1 Loaded 35 chid ids of type : 3594 +++++++++++++++++++Number RLEs to save : 11787 TO DO : save crop sub photo not yet done ! save time : 19.095043420791626 nb_obj : 19 nb_hashtags : 4 time to prepare the origin masks : 6.650109052658081 time for calcul the mask position with numpy : 0.6037037372589111 nb_pixel_total : 6362010 time to create 1 rle with new method : 0.8041110038757324 time for calcul the mask position with numpy : 0.021869182586669922 nb_pixel_total : 53739 time to create 1 rle with old method : 0.06005549430847168 time for calcul the mask position with numpy : 0.02029109001159668 nb_pixel_total : 28597 time to create 1 rle with old method : 0.03218960762023926 time for calcul the mask position with numpy : 0.0207216739654541 nb_pixel_total : 30614 time to create 1 rle with old method : 0.034204721450805664 time for calcul the mask position with numpy : 0.027904272079467773 nb_pixel_total : 26323 time to create 1 rle with old method : 0.02977752685546875 time for calcul the mask position with numpy : 0.034687042236328125 nb_pixel_total : 23219 time to create 1 rle with old method : 0.026385784149169922 time for calcul the mask position with numpy : 0.03750038146972656 nb_pixel_total : 47365 time to create 1 rle with old method : 0.0753774642944336 time for calcul the mask position with numpy : 0.0337519645690918 nb_pixel_total : 71200 time to create 1 rle with old method : 0.08020901679992676 time for calcul the mask position with numpy : 0.028764009475708008 nb_pixel_total : 23162 time to create 1 rle with old method : 0.02663254737854004 time for calcul the mask position with numpy : 0.02787923812866211 nb_pixel_total : 34109 time to create 1 rle with old method : 0.03951883316040039 time for calcul the mask position with numpy : 0.03387165069580078 nb_pixel_total : 45624 time to create 1 rle with old method : 0.05196714401245117 time for calcul the mask position with numpy : 0.021387577056884766 nb_pixel_total : 27916 time to create 1 rle with old method : 0.03223156929016113 time for calcul the mask position with numpy : 0.021481990814208984 nb_pixel_total : 23260 time to create 1 rle with old method : 0.02670001983642578 time for calcul the mask position with numpy : 0.02268838882446289 nb_pixel_total : 8029 time to create 1 rle with old method : 0.009334087371826172 time for calcul the mask position with numpy : 0.021987438201904297 nb_pixel_total : 63708 time to create 1 rle with old method : 0.0730581283569336 time for calcul the mask position with numpy : 0.022008180618286133 nb_pixel_total : 44960 time to create 1 rle with old method : 0.05162763595581055 time for calcul the mask position with numpy : 0.021996498107910156 nb_pixel_total : 66477 time to create 1 rle with old method : 0.07655477523803711 time for calcul the mask position with numpy : 0.022905349731445312 nb_pixel_total : 40282 time to create 1 rle with old method : 0.06603240966796875 time for calcul the mask position with numpy : 0.024238109588623047 nb_pixel_total : 13354 time to create 1 rle with old method : 0.015004634857177734 time for calcul the mask position with numpy : 0.021181344985961914 nb_pixel_total : 16292 time to create 1 rle with old method : 0.018580198287963867 create new chi : 2.7583699226379395 time to delete rle : 0.001621246337890625 batch 1 Loaded 39 chid ids of type : 3594 +++++++++++++++++++++++Number RLEs to save : 12066 TO DO : save crop sub photo not yet done ! save time : 2.6082749366760254 nb_obj : 19 nb_hashtags : 4 time to prepare the origin masks : 6.38376259803772 time for calcul the mask position with numpy : 0.6858484745025635 nb_pixel_total : 6084025 time to create 1 rle with new method : 0.8745322227478027 time for calcul the mask position with numpy : 0.03400301933288574 nb_pixel_total : 11375 time to create 1 rle with old method : 0.013106584548950195 time for calcul the mask position with numpy : 0.03480935096740723 nb_pixel_total : 151423 time to create 1 rle with new method : 0.5949985980987549 time for calcul the mask position with numpy : 0.03431344032287598 nb_pixel_total : 19472 time to create 1 rle with old method : 0.022754192352294922 time for calcul the mask position with numpy : 0.02206563949584961 nb_pixel_total : 126118 time to create 1 rle with old method : 0.14725041389465332 time for calcul the mask position with numpy : 0.021668434143066406 nb_pixel_total : 118559 time to create 1 rle with old method : 0.13689589500427246 time for calcul the mask position with numpy : 0.02208709716796875 nb_pixel_total : 8862 time to create 1 rle with old method : 0.010301351547241211 time for calcul the mask position with numpy : 0.0213775634765625 nb_pixel_total : 78801 time to create 1 rle with old method : 0.09079957008361816 time for calcul the mask position with numpy : 0.02195906639099121 nb_pixel_total : 83810 time to create 1 rle with old method : 0.09706902503967285 time for calcul the mask position with numpy : 0.022691726684570312 nb_pixel_total : 35066 time to create 1 rle with old method : 0.04022407531738281 time for calcul the mask position with numpy : 0.021527767181396484 nb_pixel_total : 21656 time to create 1 rle with old method : 0.025332927703857422 time for calcul the mask position with numpy : 0.021487951278686523 nb_pixel_total : 28360 time to create 1 rle with old method : 0.03266406059265137 time for calcul the mask position with numpy : 0.021912097930908203 nb_pixel_total : 67636 time to create 1 rle with old method : 0.08476138114929199 time for calcul the mask position with numpy : 0.024765491485595703 nb_pixel_total : 54972 time to create 1 rle with old method : 0.07568597793579102 time for calcul the mask position with numpy : 0.025383949279785156 nb_pixel_total : 39706 time to create 1 rle with old method : 0.05094599723815918 time for calcul the mask position with numpy : 0.022419214248657227 nb_pixel_total : 23243 time to create 1 rle with old method : 0.02717113494873047 time for calcul the mask position with numpy : 0.02343606948852539 nb_pixel_total : 19529 time to create 1 rle with old method : 0.02284836769104004 time for calcul the mask position with numpy : 0.022841453552246094 nb_pixel_total : 23246 time to create 1 rle with old method : 0.030569791793823242 time for calcul the mask position with numpy : 0.022654294967651367 nb_pixel_total : 29670 time to create 1 rle with old method : 0.0360112190246582 time for calcul the mask position with numpy : 0.025059223175048828 nb_pixel_total : 24711 time to create 1 rle with old method : 0.028443574905395508 create new chi : 3.6548280715942383 time to delete rle : 0.0017795562744140625 batch 1 Loaded 39 chid ids of type : 3594 ++++++++++++++++++++++++Number RLEs to save : 13145 TO DO : save crop sub photo not yet done ! save time : 1.4140312671661377 nb_obj : 9 nb_hashtags : 2 time to prepare the origin masks : 3.891026258468628 time for calcul the mask position with numpy : 0.3456909656524658 nb_pixel_total : 6340473 time to create 1 rle with new method : 0.587517499923706 time for calcul the mask position with numpy : 0.02182483673095703 nb_pixel_total : 210425 time to create 1 rle with new method : 0.588477611541748 time for calcul the mask position with numpy : 0.02141571044921875 nb_pixel_total : 99413 time to create 1 rle with old method : 0.11125016212463379 time for calcul the mask position with numpy : 0.022742748260498047 nb_pixel_total : 233052 time to create 1 rle with new method : 0.786761999130249 time for calcul the mask position with numpy : 0.02278733253479004 nb_pixel_total : 14354 time to create 1 rle with old method : 0.016003847122192383 time for calcul the mask position with numpy : 0.020772457122802734 nb_pixel_total : 14306 time to create 1 rle with old method : 0.01636481285095215 time for calcul the mask position with numpy : 0.02331066131591797 nb_pixel_total : 22779 time to create 1 rle with old method : 0.027103424072265625 time for calcul the mask position with numpy : 0.02995157241821289 nb_pixel_total : 9374 time to create 1 rle with old method : 0.010678529739379883 time for calcul the mask position with numpy : 0.03481721878051758 nb_pixel_total : 5225 time to create 1 rle with old method : 0.005977153778076172 time for calcul the mask position with numpy : 0.0337522029876709 nb_pixel_total : 100839 time to create 1 rle with old method : 0.11297440528869629 create new chi : 2.926821231842041 time to delete rle : 0.0012922286987304688 batch 1 Loaded 19 chid ids of type : 3594 +++++++++++++Number RLEs to save : 8025 TO DO : save crop sub photo not yet done ! save time : 1.294128656387329 nb_obj : 19 nb_hashtags : 4 time to prepare the origin masks : 6.006364345550537 time for calcul the mask position with numpy : 0.43402600288391113 nb_pixel_total : 6548763 time to create 1 rle with new method : 0.656644344329834 time for calcul the mask position with numpy : 0.01992964744567871 nb_pixel_total : 22966 time to create 1 rle with old method : 0.025838851928710938 time for calcul the mask position with numpy : 0.020545244216918945 nb_pixel_total : 89731 time to create 1 rle with old method : 0.1000525951385498 time for calcul the mask position with numpy : 0.021087646484375 nb_pixel_total : 9196 time to create 1 rle with old method : 0.010088443756103516 time for calcul the mask position with numpy : 0.020331621170043945 nb_pixel_total : 10460 time to create 1 rle with old method : 0.01193094253540039 time for calcul the mask position with numpy : 0.029321908950805664 nb_pixel_total : 23865 time to create 1 rle with old method : 0.026778697967529297 time for calcul the mask position with numpy : 0.03350329399108887 nb_pixel_total : 14566 time to create 1 rle with old method : 0.016588449478149414 time for calcul the mask position with numpy : 0.0335993766784668 nb_pixel_total : 10266 time to create 1 rle with old method : 0.011940956115722656 time for calcul the mask position with numpy : 0.03403139114379883 nb_pixel_total : 79359 time to create 1 rle with old method : 0.0912015438079834 time for calcul the mask position with numpy : 0.03457760810852051 nb_pixel_total : 54592 time to create 1 rle with old method : 0.06286263465881348 time for calcul the mask position with numpy : 0.03331327438354492 nb_pixel_total : 11851 time to create 1 rle with old method : 0.01372981071472168 time for calcul the mask position with numpy : 0.03212881088256836 nb_pixel_total : 17411 time to create 1 rle with old method : 0.020776033401489258 time for calcul the mask position with numpy : 0.022321462631225586 nb_pixel_total : 4257 time to create 1 rle with old method : 0.005087614059448242 time for calcul the mask position with numpy : 0.021645545959472656 nb_pixel_total : 6586 time to create 1 rle with old method : 0.008001565933227539 time for calcul the mask position with numpy : 0.022251605987548828 nb_pixel_total : 9463 time to create 1 rle with old method : 0.011449813842773438 time for calcul the mask position with numpy : 0.022637605667114258 nb_pixel_total : 75628 time to create 1 rle with old method : 0.08851003646850586 time for calcul the mask position with numpy : 0.022030115127563477 nb_pixel_total : 14045 time to create 1 rle with old method : 0.020200490951538086 time for calcul the mask position with numpy : 0.021963119506835938 nb_pixel_total : 24660 time to create 1 rle with old method : 0.028581619262695312 time for calcul the mask position with numpy : 0.0208742618560791 nb_pixel_total : 10386 time to create 1 rle with old method : 0.011991262435913086 time for calcul the mask position with numpy : 0.020433664321899414 nb_pixel_total : 12189 time to create 1 rle with old method : 0.013855218887329102 create new chi : 2.1925697326660156 time to delete rle : 0.0014078617095947266 batch 1 Loaded 39 chid ids of type : 3594 ++++++++++++++++++++++++Number RLEs to save : 8957 TO DO : save crop sub photo not yet done ! save time : 0.8328180313110352 nb_obj : 16 nb_hashtags : 3 time to prepare the origin masks : 6.458411931991577 time for calcul the mask position with numpy : 0.48743653297424316 nb_pixel_total : 6235340 time to create 1 rle with new method : 0.9232110977172852 time for calcul the mask position with numpy : 0.03974437713623047 nb_pixel_total : 64208 time to create 1 rle with old method : 0.07468295097351074 time for calcul the mask position with numpy : 0.03732109069824219 nb_pixel_total : 59828 time to create 1 rle with old method : 0.07444620132446289 time for calcul the mask position with numpy : 0.04623603820800781 nb_pixel_total : 15916 time to create 1 rle with old method : 0.022934913635253906 time for calcul the mask position with numpy : 0.04536175727844238 nb_pixel_total : 25214 time to create 1 rle with old method : 0.036365509033203125 time for calcul the mask position with numpy : 0.036651611328125 nb_pixel_total : 127215 time to create 1 rle with old method : 0.1470324993133545 time for calcul the mask position with numpy : 0.03464317321777344 nb_pixel_total : 15070 time to create 1 rle with old method : 0.017261743545532227 time for calcul the mask position with numpy : 0.03402876853942871 nb_pixel_total : 97519 time to create 1 rle with old method : 0.10832500457763672 time for calcul the mask position with numpy : 0.034215688705444336 nb_pixel_total : 74919 time to create 1 rle with old method : 0.08469009399414062 time for calcul the mask position with numpy : 0.020053625106811523 nb_pixel_total : 31522 time to create 1 rle with old method : 0.03564453125 time for calcul the mask position with numpy : 0.02189922332763672 nb_pixel_total : 49841 time to create 1 rle with old method : 0.05680561065673828 time for calcul the mask position with numpy : 0.022297382354736328 nb_pixel_total : 11509 time to create 1 rle with old method : 0.01842975616455078 time for calcul the mask position with numpy : 0.022522687911987305 nb_pixel_total : 19893 time to create 1 rle with old method : 0.030434608459472656 time for calcul the mask position with numpy : 0.022310256958007812 nb_pixel_total : 33584 time to create 1 rle with old method : 0.05854344367980957 time for calcul the mask position with numpy : 0.02577042579650879 nb_pixel_total : 11731 time to create 1 rle with old method : 0.013289690017700195 time for calcul the mask position with numpy : 0.021643638610839844 nb_pixel_total : 19947 time to create 1 rle with old method : 0.022856473922729492 time for calcul the mask position with numpy : 0.02364635467529297 nb_pixel_total : 156984 time to create 1 rle with new method : 1.0598118305206299 create new chi : 3.8187150955200195 time to delete rle : 0.002203702926635742 batch 1 Loaded 33 chid ids of type : 3594 ++++++++++++++++++Number RLEs to save : 10496 TO DO : save crop sub photo not yet done ! save time : 1.3793025016784668 nb_obj : 27 nb_hashtags : 4 time to prepare the origin masks : 4.12244176864624 time for calcul the mask position with numpy : 0.6288812160491943 nb_pixel_total : 5763184 time to create 1 rle with new method : 0.7604756355285645 time for calcul the mask position with numpy : 0.029288768768310547 nb_pixel_total : 34523 time to create 1 rle with old method : 0.04012918472290039 time for calcul the mask position with numpy : 0.02911376953125 nb_pixel_total : 21896 time to create 1 rle with old method : 0.025383472442626953 time for calcul the mask position with numpy : 0.028803110122680664 nb_pixel_total : 110986 time to create 1 rle with old method : 0.12794899940490723 time for calcul the mask position with numpy : 0.028942108154296875 nb_pixel_total : 6062 time to create 1 rle with old method : 0.007182598114013672 time for calcul the mask position with numpy : 0.028946876525878906 nb_pixel_total : 13536 time to create 1 rle with old method : 0.015836477279663086 time for calcul the mask position with numpy : 0.029094934463500977 nb_pixel_total : 51177 time to create 1 rle with old method : 0.05867505073547363 time for calcul the mask position with numpy : 0.03004169464111328 nb_pixel_total : 65423 time to create 1 rle with old method : 0.07714581489562988 time for calcul the mask position with numpy : 0.02855658531188965 nb_pixel_total : 32510 time to create 1 rle with old method : 0.03780770301818848 time for calcul the mask position with numpy : 0.029178619384765625 nb_pixel_total : 8610 time to create 1 rle with old method : 0.01018071174621582 time for calcul the mask position with numpy : 0.029307126998901367 nb_pixel_total : 34246 time to create 1 rle with old method : 0.04085731506347656 time for calcul the mask position with numpy : 0.0290682315826416 nb_pixel_total : 89225 time to create 1 rle with old method : 0.10065078735351562 time for calcul the mask position with numpy : 0.02910900115966797 nb_pixel_total : 21468 time to create 1 rle with old method : 0.03563189506530762 time for calcul the mask position with numpy : 0.03243875503540039 nb_pixel_total : 4049 time to create 1 rle with old method : 0.0049626827239990234 time for calcul the mask position with numpy : 0.029375076293945312 nb_pixel_total : 129616 time to create 1 rle with old method : 0.14632844924926758 time for calcul the mask position with numpy : 0.028827190399169922 nb_pixel_total : 83580 time to create 1 rle with old method : 0.11724638938903809 time for calcul the mask position with numpy : 0.029819965362548828 nb_pixel_total : 98614 time to create 1 rle with old method : 0.11481261253356934 time for calcul the mask position with numpy : 0.02848196029663086 nb_pixel_total : 43180 time to create 1 rle with old method : 0.05095100402832031 time for calcul the mask position with numpy : 0.029808759689331055 nb_pixel_total : 73785 time to create 1 rle with old method : 0.08430266380310059 time for calcul the mask position with numpy : 0.02931666374206543 nb_pixel_total : 108913 time to create 1 rle with old method : 0.12647533416748047 time for calcul the mask position with numpy : 0.02791762351989746 nb_pixel_total : 24498 time to create 1 rle with old method : 0.028004884719848633 time for calcul the mask position with numpy : 0.028535127639770508 nb_pixel_total : 7837 time to create 1 rle with old method : 0.0089874267578125 time for calcul the mask position with numpy : 0.028595685958862305 nb_pixel_total : 100695 time to create 1 rle with old method : 0.11486673355102539 time for calcul the mask position with numpy : 0.028364896774291992 nb_pixel_total : 23686 time to create 1 rle with old method : 0.027386903762817383 time for calcul the mask position with numpy : 0.02863764762878418 nb_pixel_total : 15317 time to create 1 rle with old method : 0.01796102523803711 time for calcul the mask position with numpy : 0.029330015182495117 nb_pixel_total : 40753 time to create 1 rle with old method : 0.057143449783325195 time for calcul the mask position with numpy : 0.03287148475646973 nb_pixel_total : 31570 time to create 1 rle with old method : 0.04013323783874512 time for calcul the mask position with numpy : 0.029023170471191406 nb_pixel_total : 11301 time to create 1 rle with old method : 0.013017654418945312 create new chi : 3.744130849838257 time to delete rle : 0.002575397491455078 batch 1 Loaded 55 chid ids of type : 3594 +++++++++++++++++++++++++++++Number RLEs to save : 16260 TO DO : save crop sub photo not yet done ! save time : 1.3740606307983398 nb_obj : 11 nb_hashtags : 2 time to prepare the origin masks : 4.065617084503174 time for calcul the mask position with numpy : 0.5711793899536133 nb_pixel_total : 5641260 time to create 1 rle with new method : 0.6006906032562256 time for calcul the mask position with numpy : 0.020545005798339844 nb_pixel_total : 9083 time to create 1 rle with old method : 0.010187149047851562 time for calcul the mask position with numpy : 0.021106719970703125 nb_pixel_total : 60205 time to create 1 rle with old method : 0.06892108917236328 time for calcul the mask position with numpy : 0.021861553192138672 nb_pixel_total : 105874 time to create 1 rle with old method : 0.12087297439575195 time for calcul the mask position with numpy : 0.020731210708618164 nb_pixel_total : 7769 time to create 1 rle with old method : 0.009367227554321289 time for calcul the mask position with numpy : 0.022917747497558594 nb_pixel_total : 33379 time to create 1 rle with old method : 0.04221463203430176 time for calcul the mask position with numpy : 0.023938417434692383 nb_pixel_total : 282674 time to create 1 rle with new method : 0.8378913402557373 time for calcul the mask position with numpy : 0.022960186004638672 nb_pixel_total : 4353 time to create 1 rle with old method : 0.00553441047668457 time for calcul the mask position with numpy : 0.02224254608154297 nb_pixel_total : 86189 time to create 1 rle with old method : 0.13776350021362305 time for calcul the mask position with numpy : 0.0220947265625 nb_pixel_total : 30200 time to create 1 rle with old method : 0.03471040725708008 time for calcul the mask position with numpy : 0.024655580520629883 nb_pixel_total : 384075 time to create 1 rle with new method : 0.44862866401672363 time for calcul the mask position with numpy : 0.025002717971801758 nb_pixel_total : 405179 time to create 1 rle with new method : 0.670651912689209 create new chi : 3.9069299697875977 time to delete rle : 0.0021123886108398438 batch 1 Loaded 23 chid ids of type : 3594 ++++++++++++Number RLEs to save : 10028 TO DO : save crop sub photo not yet done ! save time : 1.1303930282592773 map_output_result : {1356625852: (0.0, 'Should be the crop_list due to order', 0), 1356625849: (0.0, 'Should be the crop_list due to order', 0), 1356625846: (0.0, 'Should be the crop_list due to order', 0), 1356625842: (0.0, 'Should be the crop_list due to order', 0), 1356625822: (0.0, 'Should be the crop_list due to order', 0), 1356625767: (0.0, 'Should be the crop_list due to order', 0), 1356625765: (0.0, 'Should be the crop_list due to order', 0), 1356625753: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1356625852, 1356625849, 1356625846, 1356625842, 1356625822, 1356625767, 1356625765, 1356625753] Looping around the photos to save general results len do output : 8 /1356625852.Didn't retrieve data . /1356625849.Didn't retrieve data . /1356625846.Didn't retrieve data . /1356625842.Didn't retrieve data . /1356625822.Didn't retrieve data . /1356625767.Didn't retrieve data . /1356625765.Didn't retrieve data . /1356625753.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625852', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625849', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625846', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625842', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625822', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625767', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625765', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625753', None, None, None, None, None, '2801881') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 24 time used for this insertion : 0.013715028762817383 save_final save missing photos in datou_result : time spend for datou_step_exec : 100.66351747512817 time spend to save output : 0.014153242111206055 total time spend for step 3 : 100.67767071723938 step4:ventilate_hashtags_in_portfolio Wed May 7 09:14:54 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 22758212 get user id for portfolio 22758212 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22758212 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','environnement','carton','autre','background','metal','pet_fonce','flou','pet_clair','pehd','mal_croppe')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22758212 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','environnement','carton','autre','background','metal','pet_fonce','flou','pet_clair','pehd','mal_croppe')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22758212 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','environnement','carton','autre','background','metal','pet_fonce','flou','pet_clair','pehd','mal_croppe')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/22758403,22758404,22758405,22758406,22758407,22758408,22758409,22758410,22758411,22758412,22758413?tags=papier,environnement,carton,autre,background,metal,pet_fonce,flou,pet_clair,pehd,mal_croppe Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1356625852, 1356625849, 1356625846, 1356625842, 1356625822, 1356625767, 1356625765, 1356625753] Looping around the photos to save general results len do output : 1 /22758212. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625852', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625849', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625846', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625842', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625822', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625767', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625765', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625753', None, None, None, None, None, '2801881') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.025901079177856445 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.5687222480773926 time spend to save output : 0.026221275329589844 total time spend for step 4 : 1.5949435234069824 step5:final Wed May 7 09:14:56 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1356625852: ('0.13392497985884166',), 1356625849: ('0.13392497985884166',), 1356625846: ('0.13392497985884166',), 1356625842: ('0.13392497985884166',), 1356625822: ('0.13392497985884166',), 1356625767: ('0.13392497985884166',), 1356625765: ('0.13392497985884166',), 1356625753: ('0.13392497985884166',)} new output for save of step final : {1356625852: ('0.13392497985884166',), 1356625849: ('0.13392497985884166',), 1356625846: ('0.13392497985884166',), 1356625842: ('0.13392497985884166',), 1356625822: ('0.13392497985884166',), 1356625767: ('0.13392497985884166',), 1356625765: ('0.13392497985884166',), 1356625753: ('0.13392497985884166',)} [1356625852, 1356625849, 1356625846, 1356625842, 1356625822, 1356625767, 1356625765, 1356625753] Looping around the photos to save general results len do output : 8 /1356625852.Didn't retrieve data . /1356625849.Didn't retrieve data . /1356625846.Didn't retrieve data . /1356625842.Didn't retrieve data . /1356625822.Didn't retrieve data . /1356625767.Didn't retrieve data . /1356625765.Didn't retrieve data . /1356625753.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625852', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625849', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625846', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625842', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625822', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625767', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625765', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625753', None, None, None, None, None, '2801881') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 24 time used for this insertion : 0.012798547744750977 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.13808298110961914 time spend to save output : 0.013211965560913086 total time spend for step 5 : 0.15129494667053223 step6:blur_detection Wed May 7 09:14:56 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7.jpg resize: (2160, 3264) 1356625852 -0.21627577737611617 treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d.jpg resize: (2160, 3264) 1356625849 0.23857567975454053 treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721.jpg resize: (2160, 3264) 1356625846 -2.3765628346445715 treat image : temp/1746601827_11332_1356625842_8b910239ec8a41837a67fff24f28ca99.jpg resize: (2160, 3264) 1356625842 -1.5284553470457913 treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10.jpg resize: (2160, 3264) 1356625822 0.8647352169388296 treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567.jpg resize: (2160, 3264) 1356625767 1.0298446891144728 treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c.jpg resize: (2160, 3264) 1356625765 -3.8299438985587004 treat image : temp/1746601827_11332_1356625753_2bc477f55c2c71503045a4c3101418e1.jpg resize: (2160, 3264) 1356625753 0.026934370526004354 treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135684_0.png resize: (219, 205) 1356630543 -1.8067076800544282 treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135688_0.png resize: (338, 245) 1356630545 -0.9970095444278955 treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135683_0.png resize: (205, 95) 1356630547 -1.3894471666152668 treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135685_0.png resize: (251, 234) 1356630549 -0.9933408290399572 treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135682_0.png resize: (260, 562) 1356630550 0.6040363838128231 treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135694_0.png resize: (131, 97) 1356630552 -0.02067648543240583 treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135686_0.png resize: (89, 99) 1356630553 -0.06161555282212895 treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135697_0.png resize: (149, 227) 1356630555 -0.6208548587896936 treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135691_0.png resize: (425, 464) 1356630557 0.12175761053416895 treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135715_0.png resize: (217, 199) 1356630558 -0.7776364765990538 treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135714_0.png resize: (215, 230) 1356630560 -1.451890901573713 treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135713_0.png resize: (343, 172) 1356630562 -1.5207921380142324 treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135702_0.png resize: (369, 267) 1356630563 -1.0575276273399823 treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135705_0.png resize: (132, 91) 1356630565 -1.6818116654043187 treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135716_0.png resize: (300, 159) 1356630567 -0.008690556418260336 treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135707_0.png resize: (235, 171) 1356630568 -0.5165879711637053 treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135712_0.png resize: (199, 336) 1356630570 -0.09080131477600856 treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135706_0.png resize: (168, 229) 1356630572 -0.9118082416474604 treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135704_0.png resize: (256, 402) 1356630574 -0.04310218902473144 treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135709_0.png resize: (199, 242) 1356630575 -0.45253491217187475 treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135700_0.png resize: (104, 194) 1356630577 -1.6292941125792195 treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135699_0.png resize: (120, 210) 1356630579 -0.32768713835809743 treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135718_0.png resize: (257, 143) 1356630580 -2.054376167251886 treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135724_0.png resize: (288, 323) 1356630582 -2.2274032881717605 treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135728_0.png resize: (142, 326) 1356630584 -0.9609898881049884 treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135732_0.png resize: (460, 409) 1356630586 -2.3196597921826703 treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135731_0.png resize: (154, 131) 1356630587 -2.4040929751085542 treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135730_0.png resize: (338, 394) 1356630589 -1.7584785668944587 treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135736_0.png resize: (324, 85) 1356630591 -3.393363593475882 treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135720_0.png resize: (168, 222) 1356630592 -0.39647696692827117 treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135725_0.png resize: (310, 364) 1356630594 -0.6559683344437042 treat image : temp/1746601827_11332_1356625842_8b910239ec8a41837a67fff24f28ca99_rle_crop_3788135744_0.png resize: (454, 455) 1356630596 -2.675289043161698 treat image : temp/1746601827_11332_1356625842_8b910239ec8a41837a67fff24f28ca99_rle_crop_3788135738_0.png resize: (63, 104) 1356630597 0.6094606303093633 treat image : temp/1746601827_11332_1356625842_8b910239ec8a41837a67fff24f28ca99_rle_crop_3788135739_0.png resize: (120, 109) 1356630600 0.4898259972686955 treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135764_0.png resize: (195, 165) 1356630601 -1.6322370744645578 treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135752_0.png resize: (109, 84) 1356630602 -1.5073232229863702 treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135751_0.png resize: (115, 117) 1356630605 -1.530067467625743 treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135755_0.png resize: (119, 136) 1356630606 -0.9736342927204216 treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135754_0.png resize: (182, 125) 1356630608 -0.6951748246967948 treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135753_0.png resize: (82, 73) 1356630610 -0.7242832411796735 treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135761_0.png resize: (96, 135) 1356630611 1.0842293326430301 treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135749_0.png resize: (157, 131) 1356630614 -1.114253552201554 treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135762_0.png resize: (92, 129) 1356630615 0.8782789984790682 treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135759_0.png resize: (151, 160) 1356630616 0.4430731712129253 treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135746_0.png resize: (95, 226) 1356630618 -1.6748224205358406 treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135748_0.png resize: (202, 186) 1356630619 -0.46234741806758634 treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135767_0.png resize: (120, 153) 1356630620 -0.7331400689945626 treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135765_0.png resize: (459, 512) 1356630622 -0.49118044344757983 treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135766_0.png resize: (183, 180) 1356630623 0.030063861615573464 treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135770_0.png resize: (157, 121) 1356630624 -0.32419589321840786 treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135773_0.png resize: (292, 349) 1356630626 -0.03314540519120394 treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135776_0.png resize: (363, 467) 1356630627 0.5629155370498928 treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135769_0.png resize: (190, 155) 1356630628 -0.6978051289461976 treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135782_0.png resize: (152, 148) 1356630630 -2.4075984855608286 treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135800_0.png resize: (140, 391) 1356630631 -2.7251381174494926 treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135781_0.png resize: (442, 443) 1356630632 -3.3193920726048205 treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135786_0.png resize: (185, 204) 1356630634 -2.2194040856776485 treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135792_0.png resize: (112, 136) 1356630635 -1.8949992312205335 treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135796_0.png resize: (420, 259) 1356630637 -2.2633155354881938 treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135801_0.png resize: (378, 299) 1356630638 -0.4415259990912055 treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135791_0.png resize: (289, 346) 1356630639 -2.8240689336590292 treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135799_0.png resize: (368, 469) 1356630641 -2.242424296732166 treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135788_0.png resize: (86, 101) 1356630642 -3.56309800890002 treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135805_0.png resize: (184, 398) 1356630643 -2.361238420937725 treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135783_0.png resize: (379, 485) 1356630645 -2.1797344404559653 treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135806_0.png resize: (173, 265) 1356630646 -3.7330547557246265 treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135784_0.png resize: (186, 129) 1356630648 -2.249311564997446 treat image : temp/1746601827_11332_1356625753_2bc477f55c2c71503045a4c3101418e1_rle_crop_3788135814_0.png resize: (245, 204) 1356630649 -1.84765078275813 treat image : temp/1746601827_11332_1356625753_2bc477f55c2c71503045a4c3101418e1_rle_crop_3788135815_0.png resize: (141, 67) 1356630650 2.38209297917605 treat image : temp/1746601827_11332_1356625753_2bc477f55c2c71503045a4c3101418e1_rle_crop_3788135818_0.png resize: (125, 86) 1356630652 0.6804512368955622 treat image : temp/1746601827_11332_1356625753_2bc477f55c2c71503045a4c3101418e1_rle_crop_3788135810_0.png resize: (237, 193) 1356630653 -0.18440245360528162 treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135696_0.png resize: (254, 125) 1356630704 -2.0337893199766537 treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135717_0.png resize: (297, 300) 1356630706 -2.0233296587144514 treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135701_0.png resize: (235, 253) 1356630707 -0.8315558999615963 treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135710_0.png resize: (216, 180) 1356630708 -0.44227050011760655 treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135721_0.png resize: (182, 217) 1356630711 -2.3386440256737555 treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135722_0.png resize: (178, 192) 1356630713 -1.4506923870511008 treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135723_0.png resize: (236, 233) 1356630714 -1.85847726636057 treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135734_0.png resize: (159, 174) 1356630716 -1.345336950813989 treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135719_0.png resize: (158, 274) 1356630717 -1.8588499843968895 treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135756_0.png resize: (288, 237) 1356630719 -0.22013479790891372 treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135747_0.png resize: (126, 102) 1356630721 -0.07005731592828993 treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135794_0.png resize: (357, 296) 1356630722 -0.09504815232933554 treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135789_0.png resize: (264, 438) 1356630724 -2.1830629747654373 treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135785_0.png resize: (86, 117) 1356630726 0.8229718972680783 treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135687_0.png resize: (209, 168) 1356630853 -1.491680351664482 treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135689_0.png resize: (364, 693) 1356630854 2.190829351916497 treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135695_0.png resize: (311, 600) 1356630856 -0.6652965088138953 treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135698_0.png resize: (309, 553) 1356630857 -2.777861217653547 treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135692_0.png resize: (199, 171) 1356630858 -3.044608800652941 treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135690_0.png resize: (734, 345) 1356630859 -0.722069341113141 treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135693_0.png resize: (205, 354) 1356630860 0.06288929000947503 treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135708_0.png resize: (282, 249) 1356630861 -1.5963641437874787 treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135703_0.png resize: (241, 266) 1356630862 0.22155150925017675 treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135733_0.png resize: (549, 381) 1356630863 -1.2719612618045149 treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135729_0.png resize: (453, 386) 1356630864 -3.0319926566467674 treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135727_0.png resize: (188, 165) 1356630865 -2.79893629251991 treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135735_0.png resize: (512, 434) 1356630866 -1.6224126587676415 treat image : temp/1746601827_11332_1356625842_8b910239ec8a41837a67fff24f28ca99_rle_crop_3788135745_0.png resize: (480, 612) 1356630867 0.01718824368301426 treat image : temp/1746601827_11332_1356625842_8b910239ec8a41837a67fff24f28ca99_rle_crop_3788135743_0.png resize: (710, 464) 1356630868 -3.1939355831558944 treat image : temp/1746601827_11332_1356625842_8b910239ec8a41837a67fff24f28ca99_rle_crop_3788135740_0.png resize: (177, 159) 1356630869 -2.091811199231164 treat image : temp/1746601827_11332_1356625842_8b910239ec8a41837a67fff24f28ca99_rle_crop_3788135742_0.png resize: (134, 148) 1356630870 -0.8628427355341695 treat image : temp/1746601827_11332_1356625842_8b910239ec8a41837a67fff24f28ca99_rle_crop_3788135737_0.png resize: (264, 445) 1356630871 -0.917100715169364 treat image : temp/1746601827_11332_1356625842_8b910239ec8a41837a67fff24f28ca99_rle_crop_3788135741_0.png resize: (121, 170) 1356630872 -1.2010794311524757 treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135757_0.png resize: (237, 435) 1356630873 -0.7877644583473692 treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135750_0.png resize: (244, 464) 1356630874 -2.68726045825614 treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135763_0.png resize: (302, 431) 1356630875 -0.3259818290455518 treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135758_0.png resize: (161, 189) 1356630876 -2.347835618495358 treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135780_0.png resize: (340, 300) 1356630877 -1.6301947971966464 treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135768_0.png resize: (243, 262) 1356630878 -1.207973842912414 treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135771_0.png resize: (389, 159) 1356630879 -3.127157516729804 treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135779_0.png resize: (248, 298) 1356630880 -0.6850819295689218 treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135777_0.png resize: (209, 165) 1356630881 -1.3107138708773596 treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135772_0.png resize: (269, 302) 1356630882 0.6004663958560073 treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135774_0.png resize: (303, 383) 1356630883 1.0026116358368364 treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135775_0.png resize: (101, 205) 1356630884 -0.7877512143278957 treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135787_0.png resize: (312, 348) 1356630885 -1.7828435244302219 treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135795_0.png resize: (361, 143) 1356630886 -4.688134149534628 treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135790_0.png resize: (151, 223) 1356630887 -4.592901320422055 treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135793_0.png resize: (160, 320) 1356630888 -2.3490579325095777 treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135802_0.png resize: (199, 259) 1356630889 -2.3522682290659223 treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135803_0.png resize: (360, 357) 1356630890 -1.4347367065410146 treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135798_0.png resize: (112, 264) 1356630891 -3.2746791812088243 treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135797_0.png resize: (253, 164) 1356630892 -3.381958903108465 treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135804_0.png resize: (723, 424) 1356630893 -3.1700639551380645 treat image : temp/1746601827_11332_1356625753_2bc477f55c2c71503045a4c3101418e1_rle_crop_3788135811_0.png resize: (303, 352) 1356630894 -0.7023501089254293 treat image : temp/1746601827_11332_1356625753_2bc477f55c2c71503045a4c3101418e1_rle_crop_3788135817_0.png resize: (417, 174) 1356630895 -0.6172820841770367 treat image : temp/1746601827_11332_1356625753_2bc477f55c2c71503045a4c3101418e1_rle_crop_3788135816_0.png resize: (427, 360) 1356630896 -1.4407380769348224 treat image : temp/1746601827_11332_1356625753_2bc477f55c2c71503045a4c3101418e1_rle_crop_3788135808_0.png resize: (854, 705) 1356630897 -1.2396316136657632 treat image : temp/1746601827_11332_1356625753_2bc477f55c2c71503045a4c3101418e1_rle_crop_3788135812_0.png resize: (76, 70) 1356630898 1.3389147815274818 treat image : temp/1746601827_11332_1356625753_2bc477f55c2c71503045a4c3101418e1_rle_crop_3788135813_0.png resize: (387, 857) 1356630899 -2.3566727993850294 treat image : temp/1746601827_11332_1356625753_2bc477f55c2c71503045a4c3101418e1_rle_crop_3788135809_0.png resize: (683, 666) 1356630900 0.32426302092963893 treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135711_0.png resize: (631, 154) 1356630902 -2.1461425117832182 treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135726_0.png resize: (214, 226) 1356630903 -1.986466650195819 treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135760_0.png resize: (161, 188) 1356630904 1.1201278545916278 treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135778_0.png resize: (167, 150) 1356630905 0.004954321420707753 treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135807_0.png resize: (75, 131) 1356630906 -0.34040304690059203 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 145 time used for this insertion : 0.02004384994506836 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 145 time used for this insertion : 0.03304147720336914 save missing photos in datou_result : time spend for datou_step_exec : 28.38330578804016 time spend to save output : 0.0590510368347168 total time spend for step 6 : 28.442356824874878 step7:brightness Wed May 7 09:15:25 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7.jpg treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d.jpg treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721.jpg treat image : temp/1746601827_11332_1356625842_8b910239ec8a41837a67fff24f28ca99.jpg treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10.jpg treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567.jpg treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c.jpg treat image : temp/1746601827_11332_1356625753_2bc477f55c2c71503045a4c3101418e1.jpg treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135684_0.png treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135688_0.png treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135683_0.png treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135685_0.png treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135682_0.png treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135694_0.png treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135686_0.png treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135697_0.png treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135691_0.png treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135715_0.png treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135714_0.png treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135713_0.png treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135702_0.png treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135705_0.png treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135716_0.png treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135707_0.png treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135712_0.png treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135706_0.png treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135704_0.png treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135709_0.png treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135700_0.png treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135699_0.png treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135718_0.png treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135724_0.png treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135728_0.png treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135732_0.png treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135731_0.png treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135730_0.png treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135736_0.png treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135720_0.png treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135725_0.png treat image : temp/1746601827_11332_1356625842_8b910239ec8a41837a67fff24f28ca99_rle_crop_3788135744_0.png treat image : temp/1746601827_11332_1356625842_8b910239ec8a41837a67fff24f28ca99_rle_crop_3788135738_0.png treat image : temp/1746601827_11332_1356625842_8b910239ec8a41837a67fff24f28ca99_rle_crop_3788135739_0.png treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135764_0.png treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135752_0.png treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135751_0.png treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135755_0.png treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135754_0.png treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135753_0.png treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135761_0.png treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135749_0.png treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135762_0.png treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135759_0.png treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135746_0.png treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135748_0.png treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135767_0.png treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135765_0.png treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135766_0.png treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135770_0.png treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135773_0.png treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135776_0.png treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135769_0.png treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135782_0.png treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135800_0.png treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135781_0.png treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135786_0.png treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135792_0.png treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135796_0.png treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135801_0.png treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135791_0.png treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135799_0.png treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135788_0.png treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135805_0.png treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135783_0.png treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135806_0.png treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135784_0.png treat image : temp/1746601827_11332_1356625753_2bc477f55c2c71503045a4c3101418e1_rle_crop_3788135814_0.png treat image : temp/1746601827_11332_1356625753_2bc477f55c2c71503045a4c3101418e1_rle_crop_3788135815_0.png treat image : temp/1746601827_11332_1356625753_2bc477f55c2c71503045a4c3101418e1_rle_crop_3788135818_0.png treat image : temp/1746601827_11332_1356625753_2bc477f55c2c71503045a4c3101418e1_rle_crop_3788135810_0.png treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135696_0.png treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135717_0.png treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135701_0.png treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135710_0.png treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135721_0.png treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135722_0.png treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135723_0.png treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135734_0.png treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135719_0.png treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135756_0.png treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135747_0.png treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135794_0.png treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135789_0.png treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135785_0.png treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135687_0.png treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135689_0.png treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135695_0.png treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135698_0.png treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135692_0.png treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135690_0.png treat image : temp/1746601827_11332_1356625852_0a717679e9f82a13a1bb1ac672d473e7_rle_crop_3788135693_0.png treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135708_0.png treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135703_0.png treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135733_0.png treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135729_0.png treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135727_0.png treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135735_0.png treat image : temp/1746601827_11332_1356625842_8b910239ec8a41837a67fff24f28ca99_rle_crop_3788135745_0.png treat image : temp/1746601827_11332_1356625842_8b910239ec8a41837a67fff24f28ca99_rle_crop_3788135743_0.png treat image : temp/1746601827_11332_1356625842_8b910239ec8a41837a67fff24f28ca99_rle_crop_3788135740_0.png treat image : temp/1746601827_11332_1356625842_8b910239ec8a41837a67fff24f28ca99_rle_crop_3788135742_0.png treat image : temp/1746601827_11332_1356625842_8b910239ec8a41837a67fff24f28ca99_rle_crop_3788135737_0.png treat image : temp/1746601827_11332_1356625842_8b910239ec8a41837a67fff24f28ca99_rle_crop_3788135741_0.png treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135757_0.png treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135750_0.png treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135763_0.png treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135758_0.png treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135780_0.png treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135768_0.png treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135771_0.png treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135779_0.png treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135777_0.png treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135772_0.png treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135774_0.png treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135775_0.png treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135787_0.png treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135795_0.png treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135790_0.png treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135793_0.png treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135802_0.png treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135803_0.png treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135798_0.png treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135797_0.png treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135804_0.png treat image : temp/1746601827_11332_1356625753_2bc477f55c2c71503045a4c3101418e1_rle_crop_3788135811_0.png treat image : temp/1746601827_11332_1356625753_2bc477f55c2c71503045a4c3101418e1_rle_crop_3788135817_0.png treat image : temp/1746601827_11332_1356625753_2bc477f55c2c71503045a4c3101418e1_rle_crop_3788135816_0.png treat image : temp/1746601827_11332_1356625753_2bc477f55c2c71503045a4c3101418e1_rle_crop_3788135808_0.png treat image : temp/1746601827_11332_1356625753_2bc477f55c2c71503045a4c3101418e1_rle_crop_3788135812_0.png treat image : temp/1746601827_11332_1356625753_2bc477f55c2c71503045a4c3101418e1_rle_crop_3788135813_0.png treat image : temp/1746601827_11332_1356625753_2bc477f55c2c71503045a4c3101418e1_rle_crop_3788135809_0.png treat image : temp/1746601827_11332_1356625849_32511a27936dd006ee0f0435e1f6bc7d_rle_crop_3788135711_0.png treat image : temp/1746601827_11332_1356625846_f44e55f32075e2b21effbcc8c3cf0721_rle_crop_3788135726_0.png treat image : temp/1746601827_11332_1356625822_fe856fdd36c41000169c536602fedf10_rle_crop_3788135760_0.png treat image : temp/1746601827_11332_1356625767_3221bbd2c4b526999c225387fee1d567_rle_crop_3788135778_0.png treat image : temp/1746601827_11332_1356625765_1d8297c90c607b821f5ae72ab674605c_rle_crop_3788135807_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 145 time used for this insertion : 0.021871328353881836 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 145 time used for this insertion : 0.19855809211730957 save missing photos in datou_result : time spend for datou_step_exec : 7.490050792694092 time spend to save output : 0.22529387474060059 total time spend for step 7 : 7.715344667434692 step8:velours_tree Wed May 7 09:15:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.1773686408996582 time spend to save output : 6.461143493652344e-05 total time spend for step 8 : 0.17743325233459473 step9:send_mail_cod Wed May 7 09:15:33 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P22758212_07-05-2025_09_15_33.pdf 22758403 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227584031746602133 22758405 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227584051746602134 22758406 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227584061746602135 22758407 imagette227584071746602135 22758408 imagette227584081746602135 22758409 imagette227584091746602135 22758410 imagette227584101746602135 22758411 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227584111746602135 22758412 imagette227584121746602136 22758413 imagette227584131746602136 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=22758212 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/22758403,22758404,22758405,22758406,22758407,22758408,22758409,22758410,22758411,22758412,22758413?tags=papier,environnement,carton,autre,background,metal,pet_fonce,flou,pet_clair,pehd,mal_croppe args[1356625852] : ((1356625852, -0.21627577737611617, 492688767), (1356625852, -0.2024215560011212, 496442774), '0.13392497985884166') We are sending mail with results at report@fotonower.com args[1356625849] : ((1356625849, 0.23857567975454053, 492688767), (1356625849, 0.07300320621493943, 2107752395), '0.13392497985884166') We are sending mail with results at report@fotonower.com args[1356625846] : ((1356625846, -2.3765628346445715, 492609224), (1356625846, -0.07959788341963504, 496442774), '0.13392497985884166') We are sending mail with results at report@fotonower.com args[1356625842] : ((1356625842, -1.5284553470457913, 492688767), (1356625842, 0.16245709112305795, 2107752395), '0.13392497985884166') We are sending mail with results at report@fotonower.com args[1356625822] : ((1356625822, 0.8647352169388296, 492688767), (1356625822, 0.010497700185509785, 2107752395), '0.13392497985884166') We are sending mail with results at report@fotonower.com args[1356625767] : ((1356625767, 1.0298446891144728, 492688767), (1356625767, 0.45135399025165607, 2107752395), '0.13392497985884166') We are sending mail with results at report@fotonower.com args[1356625765] : ((1356625765, -3.8299438985587004, 492609224), (1356625765, 0.013646586064019136, 2107752395), '0.13392497985884166') We are sending mail with results at report@fotonower.com args[1356625753] : ((1356625753, 0.026934370526004354, 492688767), (1356625753, 0.3210378321392948, 2107752395), '0.13392497985884166') We are sending mail with results at report@fotonower.com refus_total : 0.13392497985884166 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=22758212 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22758212_07-05-2025_09_15_33.pdf results_Auto_P22758212_07-05-2025_09_15_33.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22758212_07-05-2025_09_15_33.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','22758212','results_Auto_P22758212_07-05-2025_09_15_33.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22758212_07-05-2025_09_15_33.pdf','pdf','','0.46','0.13392497985884166') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/22758212

https://www.fotonower.com/image?json=false&list_photos_id=1356625852
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1356625849
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1356625846
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1356625842
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1356625822
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1356625767
La photo est trop floue, merci de reprendre une photo.(avec le score = 1.0298446891144728)
https://www.fotonower.com/image?json=false&list_photos_id=1356625765
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1356625753
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 13.39%
Veuillez trouver les photos des contaminants.

exemples de contaminants: papier: https://www.fotonower.com/view/22758403?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/22758405?limit=200
exemples de contaminants: autre: https://www.fotonower.com/view/22758406?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/22758411?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22758212_07-05-2025_09_15_33.pdf.

Lien vers velours :https://www.fotonower.com/velours/22758403,22758404,22758405,22758406,22758407,22758408,22758409,22758410,22758411,22758412,22758413?tags=papier,environnement,carton,autre,background,metal,pet_fonce,flou,pet_clair,pehd,mal_croppe.


L'équipe Fotonower 202 b'' Server: nginx Date: Wed, 07 May 2025 07:15:39 GMT Content-Length: 0 Connection: close X-Message-Id: sceqo6B0S6ijf2yNZZelAw Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1356625852, 1356625849, 1356625846, 1356625842, 1356625822, 1356625767, 1356625765, 1356625753] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625852', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625849', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625846', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625842', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625822', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625767', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625765', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625753', None, None, None, None, None, '2801881') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 8 time used for this insertion : 0.013828754425048828 save_final save missing photos in datou_result : time spend for datou_step_exec : 6.146865129470825 time spend to save output : 0.014215469360351562 total time spend for step 9 : 6.161080598831177 step10:split_time_score Wed May 7 09:15:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('08', 8),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 07052025 22758212 Nombre de photos uploadées : 8 / 23040 (0%) 07052025 22758212 Nombre de photos taguées (types de déchets): 0 / 8 (0%) 07052025 22758212 Nombre de photos taguées (volume) : 0 / 8 (0%) elapsed_time : load_data_split_time_score 1.1920928955078125e-06 elapsed_time : order_list_meta_photo_and_scores 5.0067901611328125e-06 ???????? elapsed_time : fill_and_build_computed_from_old_data 0.00045228004455566406 elapsed_time : insert_dashboard_record_day_entry 0.03082561492919922 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.11345237892610745 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22755885_07-05-2025_08_15_57.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22755885 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22755885 AND mptpi.`type`=3594 To do Qualite : 0.13392497985884166 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22758212_07-05-2025_09_15_33.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22758212 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22758212 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'07052025': {'nb_upload': 8, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1356625852, 1356625849, 1356625846, 1356625842, 1356625822, 1356625767, 1356625765, 1356625753] Looping around the photos to save general results len do output : 1 /22758212Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625852', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625849', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625846', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625842', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625822', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625767', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625765', None, None, None, None, None, '2801881') ('3318', None, None, None, None, None, None, None, '2801881') ('3318', '22758212', '1356625753', None, None, None, None, None, '2801881') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.023536205291748047 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.3305819034576416 time spend to save output : 0.023759126663208008 total time spend for step 10 : 0.3543410301208496 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 8 set_done_treatment 124.74user 101.72system 5:15.33elapsed 71%CPU (0avgtext+0avgdata 5178896maxresident)k 540656inputs+71280outputs (262major+11332488minor)pagefaults 0swaps