python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 612518 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['2994566'] with mtr_portfolio_ids : ['23497861'] and first list_photo_ids : [] new path : /proc/612518/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 5 ; length of list_pids : 5 ; length of list_args : 5 time to download the photos : 0.94586181640625 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Sat May 31 15:10:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10733 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-05-31 15:10:31.873294: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-05-31 15:10:31.899455: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492915000 Hz 2025-05-31 15:10:31.901456: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f2e8c000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-05-31 15:10:31.901522: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-05-31 15:10:31.905346: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-05-31 15:10:32.035436: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xa8e2da0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-05-31 15:10:32.035499: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-05-31 15:10:32.036814: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-31 15:10:32.037298: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-31 15:10:32.040360: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-31 15:10:32.043435: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-31 15:10:32.044059: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-31 15:10:32.046794: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-31 15:10:32.048177: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-31 15:10:32.053630: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-31 15:10:32.055620: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-31 15:10:32.055718: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-31 15:10:32.056558: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-31 15:10:32.056576: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-31 15:10:32.056585: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-31 15:10:32.057944: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9803 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-05-31 15:10:32.337663: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-31 15:10:32.337820: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-31 15:10:32.337844: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-31 15:10:32.337865: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-31 15:10:32.337885: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-31 15:10:32.337904: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-31 15:10:32.337926: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-31 15:10:32.337955: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-31 15:10:32.339686: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-31 15:10:32.341432: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-31 15:10:32.341632: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-31 15:10:32.341662: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-31 15:10:32.341685: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-31 15:10:32.341707: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-31 15:10:32.341730: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-31 15:10:32.341752: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-31 15:10:32.341775: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-31 15:10:32.343547: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-31 15:10:32.343632: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-31 15:10:32.343645: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-31 15:10:32.343655: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-31 15:10:32.345729: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9803 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-05-31 15:10:40.436903: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-31 15:10:40.630244: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 5 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 46 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 2.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 51 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 6.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 48 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 2.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 33 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 6.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 13 Detection mask done ! Trying to reset tf kernel 613234 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1667 tf kernel not reseted sub process len(results) : 5 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 5 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 6956 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0014300346374511719 nb_pixel_total : 48006 time to create 1 rle with old method : 0.058469533920288086 length of segment : 297 time for calcul the mask position with numpy : 0.0003209114074707031 nb_pixel_total : 15203 time to create 1 rle with old method : 0.01733088493347168 length of segment : 106 time for calcul the mask position with numpy : 0.0004642009735107422 nb_pixel_total : 20076 time to create 1 rle with old method : 0.023152828216552734 length of segment : 178 time for calcul the mask position with numpy : 0.0013759136199951172 nb_pixel_total : 67617 time to create 1 rle with old method : 0.0753169059753418 length of segment : 317 time for calcul the mask position with numpy : 0.003009796142578125 nb_pixel_total : 106530 time to create 1 rle with old method : 0.12150931358337402 length of segment : 780 time for calcul the mask position with numpy : 0.0006549358367919922 nb_pixel_total : 23287 time to create 1 rle with old method : 0.027065038681030273 length of segment : 245 time for calcul the mask position with numpy : 0.0009062290191650391 nb_pixel_total : 58224 time to create 1 rle with old method : 0.06790351867675781 length of segment : 326 time for calcul the mask position with numpy : 0.00225830078125 nb_pixel_total : 111926 time to create 1 rle with old method : 0.13019227981567383 length of segment : 522 time for calcul the mask position with numpy : 0.0002541542053222656 nb_pixel_total : 6851 time to create 1 rle with old method : 0.008430719375610352 length of segment : 98 time for calcul the mask position with numpy : 0.0008471012115478516 nb_pixel_total : 41478 time to create 1 rle with old method : 0.048974037170410156 length of segment : 194 time for calcul the mask position with numpy : 0.0005354881286621094 nb_pixel_total : 24902 time to create 1 rle with old method : 0.032811880111694336 length of segment : 265 time for calcul the mask position with numpy : 0.0006151199340820312 nb_pixel_total : 28241 time to create 1 rle with old method : 0.03449130058288574 length of segment : 218 time for calcul the mask position with numpy : 0.0003600120544433594 nb_pixel_total : 17696 time to create 1 rle with old method : 0.02103567123413086 length of segment : 169 time for calcul the mask position with numpy : 0.0030481815338134766 nb_pixel_total : 128089 time to create 1 rle with old method : 0.14924383163452148 length of segment : 661 time for calcul the mask position with numpy : 0.0012507438659667969 nb_pixel_total : 52121 time to create 1 rle with old method : 0.06303167343139648 length of segment : 421 time for calcul the mask position with numpy : 0.0006952285766601562 nb_pixel_total : 37338 time to create 1 rle with old method : 0.04370760917663574 length of segment : 314 time for calcul the mask position with numpy : 0.0004706382751464844 nb_pixel_total : 25432 time to create 1 rle with old method : 0.030059814453125 length of segment : 222 time for calcul the mask position with numpy : 0.0018579959869384766 nb_pixel_total : 118089 time to create 1 rle with old method : 0.13532400131225586 length of segment : 540 time for calcul the mask position with numpy : 0.0007779598236083984 nb_pixel_total : 36099 time to create 1 rle with old method : 0.0420832633972168 length of segment : 224 time for calcul the mask position with numpy : 0.0003554821014404297 nb_pixel_total : 20346 time to create 1 rle with old method : 0.023753643035888672 length of segment : 100 time for calcul the mask position with numpy : 0.0017867088317871094 nb_pixel_total : 121797 time to create 1 rle with old method : 0.13716721534729004 length of segment : 455 time for calcul the mask position with numpy : 0.000986337661743164 nb_pixel_total : 55758 time to create 1 rle with old method : 0.06393218040466309 length of segment : 279 time for calcul the mask position with numpy : 0.0003867149353027344 nb_pixel_total : 17909 time to create 1 rle with old method : 0.020849227905273438 length of segment : 176 time for calcul the mask position with numpy : 0.0005042552947998047 nb_pixel_total : 7656 time to create 1 rle with old method : 0.009033679962158203 length of segment : 167 time for calcul the mask position with numpy : 0.0024712085723876953 nb_pixel_total : 44282 time to create 1 rle with old method : 0.050659894943237305 length of segment : 426 time for calcul the mask position with numpy : 0.0011820793151855469 nb_pixel_total : 73216 time to create 1 rle with old method : 0.08204317092895508 length of segment : 324 time for calcul the mask position with numpy : 0.007805347442626953 nb_pixel_total : 166470 time to create 1 rle with new method : 0.012155771255493164 length of segment : 674 time for calcul the mask position with numpy : 0.0034847259521484375 nb_pixel_total : 61840 time to create 1 rle with old method : 0.06963634490966797 length of segment : 368 time for calcul the mask position with numpy : 0.0012807846069335938 nb_pixel_total : 23076 time to create 1 rle with old method : 0.026483535766601562 length of segment : 286 time for calcul the mask position with numpy : 0.0008897781372070312 nb_pixel_total : 21582 time to create 1 rle with old method : 0.024783849716186523 length of segment : 240 time for calcul the mask position with numpy : 0.005536556243896484 nb_pixel_total : 123293 time to create 1 rle with old method : 0.1616072654724121 length of segment : 538 time for calcul the mask position with numpy : 0.0017864704132080078 nb_pixel_total : 45576 time to create 1 rle with old method : 0.05165505409240723 length of segment : 329 time for calcul the mask position with numpy : 0.0004420280456542969 nb_pixel_total : 8390 time to create 1 rle with old method : 0.013377904891967773 length of segment : 69 time for calcul the mask position with numpy : 0.00167083740234375 nb_pixel_total : 41383 time to create 1 rle with old method : 0.0486445426940918 length of segment : 210 time for calcul the mask position with numpy : 0.0005307197570800781 nb_pixel_total : 13004 time to create 1 rle with old method : 0.01517939567565918 length of segment : 190 time for calcul the mask position with numpy : 0.0018720626831054688 nb_pixel_total : 43929 time to create 1 rle with old method : 0.05054831504821777 length of segment : 321 time for calcul the mask position with numpy : 0.004862546920776367 nb_pixel_total : 104367 time to create 1 rle with old method : 0.11483263969421387 length of segment : 477 time for calcul the mask position with numpy : 0.00025916099548339844 nb_pixel_total : 7218 time to create 1 rle with old method : 0.00803685188293457 length of segment : 91 time for calcul the mask position with numpy : 0.0002818107604980469 nb_pixel_total : 8398 time to create 1 rle with old method : 0.009700775146484375 length of segment : 94 time for calcul the mask position with numpy : 0.0002586841583251953 nb_pixel_total : 7682 time to create 1 rle with old method : 0.00850820541381836 length of segment : 179 time for calcul the mask position with numpy : 0.0019567012786865234 nb_pixel_total : 61205 time to create 1 rle with old method : 0.07868504524230957 length of segment : 257 time for calcul the mask position with numpy : 0.0028429031372070312 nb_pixel_total : 62966 time to create 1 rle with old method : 0.07267951965332031 length of segment : 395 time for calcul the mask position with numpy : 0.001941680908203125 nb_pixel_total : 38759 time to create 1 rle with old method : 0.04370594024658203 length of segment : 268 time for calcul the mask position with numpy : 0.0012717247009277344 nb_pixel_total : 30975 time to create 1 rle with old method : 0.0355226993560791 length of segment : 208 time for calcul the mask position with numpy : 0.0025463104248046875 nb_pixel_total : 56164 time to create 1 rle with old method : 0.06223893165588379 length of segment : 322 time for calcul the mask position with numpy : 0.0035119056701660156 nb_pixel_total : 72902 time to create 1 rle with old method : 0.0812387466430664 length of segment : 815 time for calcul the mask position with numpy : 0.0075283050537109375 nb_pixel_total : 208980 time to create 1 rle with new method : 0.012860298156738281 length of segment : 423 time for calcul the mask position with numpy : 0.0018239021301269531 nb_pixel_total : 40399 time to create 1 rle with old method : 0.04495692253112793 length of segment : 275 time for calcul the mask position with numpy : 0.004705190658569336 nb_pixel_total : 127479 time to create 1 rle with old method : 0.14156413078308105 length of segment : 397 time for calcul the mask position with numpy : 0.004693746566772461 nb_pixel_total : 126835 time to create 1 rle with old method : 0.14172887802124023 length of segment : 383 time for calcul the mask position with numpy : 0.004591703414916992 nb_pixel_total : 123693 time to create 1 rle with old method : 0.13818144798278809 length of segment : 462 time for calcul the mask position with numpy : 0.005705595016479492 nb_pixel_total : 165988 time to create 1 rle with new method : 0.011182069778442383 length of segment : 528 time for calcul the mask position with numpy : 0.0037615299224853516 nb_pixel_total : 93688 time to create 1 rle with old method : 0.10285544395446777 length of segment : 524 time for calcul the mask position with numpy : 0.0010211467742919922 nb_pixel_total : 28708 time to create 1 rle with old method : 0.03181815147399902 length of segment : 196 time for calcul the mask position with numpy : 0.0009069442749023438 nb_pixel_total : 15992 time to create 1 rle with old method : 0.01830267906188965 length of segment : 212 time for calcul the mask position with numpy : 0.004652738571166992 nb_pixel_total : 96948 time to create 1 rle with old method : 0.10787200927734375 length of segment : 763 time for calcul the mask position with numpy : 0.0034182071685791016 nb_pixel_total : 97729 time to create 1 rle with old method : 0.10961365699768066 length of segment : 623 time for calcul the mask position with numpy : 0.0014812946319580078 nb_pixel_total : 35920 time to create 1 rle with old method : 0.04080629348754883 length of segment : 209 time for calcul the mask position with numpy : 0.00047969818115234375 nb_pixel_total : 12208 time to create 1 rle with old method : 0.014244318008422852 length of segment : 115 time for calcul the mask position with numpy : 0.0030510425567626953 nb_pixel_total : 75505 time to create 1 rle with old method : 0.08649778366088867 length of segment : 268 time for calcul the mask position with numpy : 0.0005626678466796875 nb_pixel_total : 10915 time to create 1 rle with old method : 0.012602090835571289 length of segment : 119 time for calcul the mask position with numpy : 0.0019643306732177734 nb_pixel_total : 57142 time to create 1 rle with old method : 0.06346273422241211 length of segment : 404 time for calcul the mask position with numpy : 0.000469207763671875 nb_pixel_total : 8000 time to create 1 rle with old method : 0.009169816970825195 length of segment : 85 time for calcul the mask position with numpy : 0.0002925395965576172 nb_pixel_total : 6307 time to create 1 rle with old method : 0.007522106170654297 length of segment : 102 time for calcul the mask position with numpy : 0.002419710159301758 nb_pixel_total : 71710 time to create 1 rle with old method : 0.08189105987548828 length of segment : 361 time for calcul the mask position with numpy : 0.0033926963806152344 nb_pixel_total : 74153 time to create 1 rle with old method : 0.08037161827087402 length of segment : 407 time for calcul the mask position with numpy : 0.00867605209350586 nb_pixel_total : 170940 time to create 1 rle with new method : 0.016634225845336914 length of segment : 571 time for calcul the mask position with numpy : 0.005476713180541992 nb_pixel_total : 112537 time to create 1 rle with old method : 0.12400603294372559 length of segment : 400 time for calcul the mask position with numpy : 0.004638195037841797 nb_pixel_total : 83768 time to create 1 rle with old method : 0.11321902275085449 length of segment : 335 time for calcul the mask position with numpy : 0.0014290809631347656 nb_pixel_total : 29492 time to create 1 rle with old method : 0.03344893455505371 length of segment : 280 time for calcul the mask position with numpy : 0.0007183551788330078 nb_pixel_total : 17313 time to create 1 rle with old method : 0.01979207992553711 length of segment : 161 time for calcul the mask position with numpy : 0.006520271301269531 nb_pixel_total : 244151 time to create 1 rle with new method : 0.013330459594726562 length of segment : 663 time for calcul the mask position with numpy : 0.0016329288482666016 nb_pixel_total : 45392 time to create 1 rle with old method : 0.05159187316894531 length of segment : 181 time for calcul the mask position with numpy : 0.002003192901611328 nb_pixel_total : 64343 time to create 1 rle with old method : 0.07308411598205566 length of segment : 241 time for calcul the mask position with numpy : 0.0010786056518554688 nb_pixel_total : 31957 time to create 1 rle with old method : 0.034926414489746094 length of segment : 208 time for calcul the mask position with numpy : 0.0006234645843505859 nb_pixel_total : 14337 time to create 1 rle with old method : 0.016143083572387695 length of segment : 291 time for calcul the mask position with numpy : 0.0005323886871337891 nb_pixel_total : 14382 time to create 1 rle with old method : 0.01630711555480957 length of segment : 112 time for calcul the mask position with numpy : 0.0006062984466552734 nb_pixel_total : 14191 time to create 1 rle with old method : 0.015985488891601562 length of segment : 192 time for calcul the mask position with numpy : 0.0005784034729003906 nb_pixel_total : 13410 time to create 1 rle with old method : 0.015047788619995117 length of segment : 149 time for calcul the mask position with numpy : 0.004286766052246094 nb_pixel_total : 144535 time to create 1 rle with old method : 0.15468406677246094 length of segment : 478 time for calcul the mask position with numpy : 0.003128528594970703 nb_pixel_total : 84460 time to create 1 rle with old method : 0.09494471549987793 length of segment : 557 time for calcul the mask position with numpy : 0.0015871524810791016 nb_pixel_total : 38308 time to create 1 rle with old method : 0.043791770935058594 length of segment : 268 time for calcul the mask position with numpy : 0.0022830963134765625 nb_pixel_total : 64725 time to create 1 rle with old method : 0.07734179496765137 length of segment : 310 time for calcul the mask position with numpy : 0.0028634071350097656 nb_pixel_total : 90925 time to create 1 rle with old method : 0.1024477481842041 length of segment : 545 time for calcul the mask position with numpy : 0.003088235855102539 nb_pixel_total : 111184 time to create 1 rle with old method : 0.12988710403442383 length of segment : 427 time for calcul the mask position with numpy : 0.0003452301025390625 nb_pixel_total : 8231 time to create 1 rle with old method : 0.00962972640991211 length of segment : 91 time for calcul the mask position with numpy : 0.0028858184814453125 nb_pixel_total : 98809 time to create 1 rle with old method : 0.11128115653991699 length of segment : 334 time for calcul the mask position with numpy : 0.0022912025451660156 nb_pixel_total : 74016 time to create 1 rle with old method : 0.0868370532989502 length of segment : 302 time for calcul the mask position with numpy : 0.00402522087097168 nb_pixel_total : 164342 time to create 1 rle with new method : 0.008841753005981445 length of segment : 704 time for calcul the mask position with numpy : 0.0016715526580810547 nb_pixel_total : 29101 time to create 1 rle with old method : 0.03470754623413086 length of segment : 180 time for calcul the mask position with numpy : 0.00040435791015625 nb_pixel_total : 12901 time to create 1 rle with old method : 0.014741897583007812 length of segment : 127 time for calcul the mask position with numpy : 0.0005121231079101562 nb_pixel_total : 14778 time to create 1 rle with old method : 0.017101049423217773 length of segment : 198 time for calcul the mask position with numpy : 0.0005629062652587891 nb_pixel_total : 14881 time to create 1 rle with old method : 0.017136812210083008 length of segment : 159 time for calcul the mask position with numpy : 0.0009996891021728516 nb_pixel_total : 35860 time to create 1 rle with old method : 0.04038500785827637 length of segment : 245 time for calcul the mask position with numpy : 0.00018310546875 nb_pixel_total : 10683 time to create 1 rle with old method : 0.012125492095947266 length of segment : 128 time for calcul the mask position with numpy : 0.0009393692016601562 nb_pixel_total : 45870 time to create 1 rle with old method : 0.05185842514038086 length of segment : 231 time for calcul the mask position with numpy : 0.0005271434783935547 nb_pixel_total : 18228 time to create 1 rle with old method : 0.02105402946472168 length of segment : 153 time for calcul the mask position with numpy : 0.07106161117553711 nb_pixel_total : 3459494 time to create 1 rle with new method : 0.11884808540344238 length of segment : 2515 time for calcul the mask position with numpy : 0.0013453960418701172 nb_pixel_total : 77291 time to create 1 rle with old method : 0.0941925048828125 length of segment : 282 time spent for convertir_results : 10.826135873794556 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 99 chid ids of type : 3594 Number RLEs to save : 33229 save missing photos in datou_result : time spend for datou_step_exec : 50.58699107170105 time spend to save output : 2.099874496459961 total time spend for step 1 : 52.68686556816101 step2:crop_condition Sat May 31 15:11:21 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 5 ! batch 1 Loaded 99 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 68 About to insert : list_path_to_insert length 68 new photo from crops ! About to upload 68 photos upload in portfolio : 3736932 init cache_photo without model_param we have 68 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1748697099_612518 we have uploaded 68 photos in the portfolio 3736932 time of upload the photos Elapsed time : 16.882649183273315 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 3736932 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1748697117_612518 we have uploaded 5 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.571953296661377 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1748697120_612518 we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.7113621234893799 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 21 About to insert : list_path_to_insert length 21 new photo from crops ! About to upload 21 photos upload in portfolio : 3736932 init cache_photo without model_param we have 21 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1748697142_612518 we have uploaded 21 photos in the portfolio 3736932 time of upload the photos Elapsed time : 5.897497892379761 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 3736932 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1748697149_612518 we have uploaded 3 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.9181973934173584 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1748697151_612518 we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 3.661987066268921 we have finished the crop for the class : pet_fonce delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1362198073, 1362198068, 1362198066, 1362198064, 1362197939] Looping around the photos to save general results len do output : 99 /1362226449Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226450Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226451Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226452Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226453Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226454Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226455Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226456Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226457Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226458Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226459Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226460Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226461Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226462Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226463Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226464Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226465Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226466Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226467Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226468Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226469Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226470Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226471Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226472Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226473Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226474Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226475Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226476Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226477Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226478Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226479Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226480Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226481Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226482Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226483Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226484Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226485Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226486Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226487Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226488Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226490Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226491Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226492Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226494Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226495Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226496Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226498Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226499Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226500Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226502Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226503Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226504Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226506Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226507Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226508Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226510Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226511Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226513Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226515Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226516Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226518Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226520Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226521Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226522Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226524Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226525Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226526Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226527Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226543Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226544Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226546Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226547Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226548Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226560Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226853Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226855Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226856Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226858Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226860Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226861Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226863Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226866Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226867Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226869Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226871Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226872Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226874Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226875Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226876Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226879Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226880Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226881Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226883Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226884Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226885Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226900Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226901Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226903Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1362226951Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2994566') ('3318', '23497861', '1362198073', None, None, None, None, None, '2994566') ('3318', None, None, None, None, None, None, None, '2994566') ('3318', '23497861', '1362198068', None, None, None, None, None, '2994566') ('3318', None, None, None, None, None, None, None, '2994566') ('3318', '23497861', '1362198066', None, None, None, None, None, '2994566') ('3318', None, None, None, None, None, None, None, '2994566') ('3318', '23497861', '1362198064', None, None, None, None, None, '2994566') ('3318', None, None, None, None, None, None, None, '2994566') ('3318', '23497861', '1362197939', None, None, None, None, None, '2994566') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 302 time used for this insertion : 0.03439617156982422 save_final save missing photos in datou_result : time spend for datou_step_exec : 73.36158585548401 time spend to save output : 0.03718233108520508 total time spend for step 2 : 73.39876818656921 step3:rle_unique_nms_with_priority Sat May 31 15:12:35 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 99 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 23 nb_hashtags : 4 time to prepare the origin masks : 3.4121429920196533 time for calcul the mask position with numpy : 0.5699200630187988 nb_pixel_total : 7111385 time to create 1 rle with new method : 0.6627271175384521 time for calcul the mask position with numpy : 0.023251771926879883 nb_pixel_total : 17909 time to create 1 rle with old method : 0.01928114891052246 time for calcul the mask position with numpy : 0.0225827693939209 nb_pixel_total : 55758 time to create 1 rle with old method : 0.07400298118591309 time for calcul the mask position with numpy : 0.026358604431152344 nb_pixel_total : 121797 time to create 1 rle with old method : 0.13869023323059082 time for calcul the mask position with numpy : 0.023420095443725586 nb_pixel_total : 20346 time to create 1 rle with old method : 0.02282261848449707 time for calcul the mask position with numpy : 0.023988723754882812 nb_pixel_total : 36099 time to create 1 rle with old method : 0.040686845779418945 time for calcul the mask position with numpy : 0.025397062301635742 nb_pixel_total : 118089 time to create 1 rle with old method : 0.1284322738647461 time for calcul the mask position with numpy : 0.0231473445892334 nb_pixel_total : 25432 time to create 1 rle with old method : 0.0282135009765625 time for calcul the mask position with numpy : 0.024427175521850586 nb_pixel_total : 37338 time to create 1 rle with old method : 0.040683746337890625 time for calcul the mask position with numpy : 0.02349996566772461 nb_pixel_total : 52121 time to create 1 rle with old method : 0.0569767951965332 time for calcul the mask position with numpy : 0.02417302131652832 nb_pixel_total : 128089 time to create 1 rle with old method : 0.13628578186035156 time for calcul the mask position with numpy : 0.024112701416015625 nb_pixel_total : 17696 time to create 1 rle with old method : 0.019306659698486328 time for calcul the mask position with numpy : 0.024227142333984375 nb_pixel_total : 28241 time to create 1 rle with old method : 0.03175163269042969 time for calcul the mask position with numpy : 0.02396106719970703 nb_pixel_total : 24902 time to create 1 rle with old method : 0.027593135833740234 time for calcul the mask position with numpy : 0.02518439292907715 nb_pixel_total : 41478 time to create 1 rle with old method : 0.045766592025756836 time for calcul the mask position with numpy : 0.026018857955932617 nb_pixel_total : 6851 time to create 1 rle with old method : 0.007663249969482422 time for calcul the mask position with numpy : 0.024210453033447266 nb_pixel_total : 111926 time to create 1 rle with old method : 0.12169146537780762 time for calcul the mask position with numpy : 0.02469801902770996 nb_pixel_total : 58224 time to create 1 rle with old method : 0.06395101547241211 time for calcul the mask position with numpy : 0.02614140510559082 nb_pixel_total : 23287 time to create 1 rle with old method : 0.025841236114501953 time for calcul the mask position with numpy : 0.02673196792602539 nb_pixel_total : 106530 time to create 1 rle with old method : 0.11550068855285645 time for calcul the mask position with numpy : 0.025719642639160156 nb_pixel_total : 67617 time to create 1 rle with old method : 0.07262396812438965 time for calcul the mask position with numpy : 0.024862051010131836 nb_pixel_total : 20076 time to create 1 rle with old method : 0.02317500114440918 time for calcul the mask position with numpy : 0.025524377822875977 nb_pixel_total : 15203 time to create 1 rle with old method : 0.024484872817993164 time for calcul the mask position with numpy : 0.028554439544677734 nb_pixel_total : 48006 time to create 1 rle with old method : 0.05336594581604004 create new chi : 3.1590664386749268 time to delete rle : 0.01941227912902832 batch 1 Loaded 47 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++Number RLEs to save : 16374 TO DO : save crop sub photo not yet done ! save time : 1.0369203090667725 nb_obj : 23 nb_hashtags : 4 time to prepare the origin masks : 3.2122983932495117 time for calcul the mask position with numpy : 0.4908781051635742 nb_pixel_total : 7171280 time to create 1 rle with new method : 0.4577610492706299 time for calcul the mask position with numpy : 0.023840665817260742 nb_pixel_total : 72544 time to create 1 rle with old method : 0.07904577255249023 time for calcul the mask position with numpy : 0.0272524356842041 nb_pixel_total : 56164 time to create 1 rle with old method : 0.07350492477416992 time for calcul the mask position with numpy : 0.031049728393554688 nb_pixel_total : 30975 time to create 1 rle with old method : 0.03711199760437012 time for calcul the mask position with numpy : 0.02457404136657715 nb_pixel_total : 38759 time to create 1 rle with old method : 0.04274725914001465 time for calcul the mask position with numpy : 0.023654937744140625 nb_pixel_total : 62966 time to create 1 rle with old method : 0.07007646560668945 time for calcul the mask position with numpy : 0.025414466857910156 nb_pixel_total : 61205 time to create 1 rle with old method : 0.06804084777832031 time for calcul the mask position with numpy : 0.023799657821655273 nb_pixel_total : 7388 time to create 1 rle with old method : 0.008254766464233398 time for calcul the mask position with numpy : 0.023743391036987305 nb_pixel_total : 8232 time to create 1 rle with old method : 0.009253263473510742 time for calcul the mask position with numpy : 0.0251009464263916 nb_pixel_total : 7218 time to create 1 rle with old method : 0.008105993270874023 time for calcul the mask position with numpy : 0.024726390838623047 nb_pixel_total : 104367 time to create 1 rle with old method : 0.12430548667907715 time for calcul the mask position with numpy : 0.02593398094177246 nb_pixel_total : 43929 time to create 1 rle with old method : 0.059416770935058594 time for calcul the mask position with numpy : 0.027333736419677734 nb_pixel_total : 13004 time to create 1 rle with old method : 0.014811038970947266 time for calcul the mask position with numpy : 0.025308609008789062 nb_pixel_total : 41383 time to create 1 rle with old method : 0.046164512634277344 time for calcul the mask position with numpy : 0.025460481643676758 nb_pixel_total : 8390 time to create 1 rle with old method : 0.009421586990356445 time for calcul the mask position with numpy : 0.025777816772460938 nb_pixel_total : 45181 time to create 1 rle with old method : 0.050314903259277344 time for calcul the mask position with numpy : 0.029763221740722656 nb_pixel_total : 123293 time to create 1 rle with old method : 0.1346874237060547 time for calcul the mask position with numpy : 0.025038480758666992 nb_pixel_total : 21582 time to create 1 rle with old method : 0.02395486831665039 time for calcul the mask position with numpy : 0.02407693862915039 nb_pixel_total : 23076 time to create 1 rle with old method : 0.025707721710205078 time for calcul the mask position with numpy : 0.02524566650390625 nb_pixel_total : 61840 time to create 1 rle with old method : 0.0695490837097168 time for calcul the mask position with numpy : 0.023733854293823242 nb_pixel_total : 166470 time to create 1 rle with new method : 0.1615433692932129 time for calcul the mask position with numpy : 0.023839950561523438 nb_pixel_total : 73216 time to create 1 rle with old method : 0.07622861862182617 time for calcul the mask position with numpy : 0.0227811336517334 nb_pixel_total : 44282 time to create 1 rle with old method : 0.04700279235839844 time for calcul the mask position with numpy : 0.02391815185546875 nb_pixel_total : 7656 time to create 1 rle with old method : 0.008293390274047852 create new chi : 2.8387339115142822 time to delete rle : 0.0017659664154052734 batch 1 Loaded 47 chid ids of type : 3594 ++++++++++++++++++++++++++++++Number RLEs to save : 16412 TO DO : save crop sub photo not yet done ! save time : 1.0056755542755127 nb_obj : 27 nb_hashtags : 3 time to prepare the origin masks : 4.7628090381622314 time for calcul the mask position with numpy : 0.07445311546325684 nb_pixel_total : 6119579 time to create 1 rle with new method : 0.1581418514251709 time for calcul the mask position with numpy : 0.04081130027770996 nb_pixel_total : 28708 time to create 1 rle with old method : 0.03074336051940918 time for calcul the mask position with numpy : 0.03407549858093262 nb_pixel_total : 97729 time to create 1 rle with old method : 0.10972285270690918 time for calcul the mask position with numpy : 0.03321337699890137 nb_pixel_total : 74153 time to create 1 rle with old method : 0.08013415336608887 time for calcul the mask position with numpy : 0.03236103057861328 nb_pixel_total : 127479 time to create 1 rle with old method : 0.13939666748046875 time for calcul the mask position with numpy : 0.03441643714904785 nb_pixel_total : 208980 time to create 1 rle with new method : 0.14379358291625977 time for calcul the mask position with numpy : 0.03272271156311035 nb_pixel_total : 112537 time to create 1 rle with old method : 0.12114858627319336 time for calcul the mask position with numpy : 0.033675193786621094 nb_pixel_total : 244059 time to create 1 rle with new method : 0.14246416091918945 time for calcul the mask position with numpy : 0.0320582389831543 nb_pixel_total : 71710 time to create 1 rle with old method : 0.07652115821838379 time for calcul the mask position with numpy : 0.03207540512084961 nb_pixel_total : 17313 time to create 1 rle with old method : 0.01908111572265625 time for calcul the mask position with numpy : 0.032656192779541016 nb_pixel_total : 35920 time to create 1 rle with old method : 0.0387578010559082 time for calcul the mask position with numpy : 0.03427314758300781 nb_pixel_total : 170940 time to create 1 rle with new method : 0.14278435707092285 time for calcul the mask position with numpy : 0.031583309173583984 nb_pixel_total : 93688 time to create 1 rle with old method : 0.10834312438964844 time for calcul the mask position with numpy : 0.032845258712768555 nb_pixel_total : 40399 time to create 1 rle with old method : 0.048303842544555664 time for calcul the mask position with numpy : 0.031861305236816406 nb_pixel_total : 96948 time to create 1 rle with old method : 0.11522698402404785 time for calcul the mask position with numpy : 0.031746625900268555 nb_pixel_total : 15992 time to create 1 rle with old method : 0.01673150062561035 time for calcul the mask position with numpy : 0.03156256675720215 nb_pixel_total : 8000 time to create 1 rle with old method : 0.008435487747192383 time for calcul the mask position with numpy : 0.032560110092163086 nb_pixel_total : 165988 time to create 1 rle with new method : 0.13926434516906738 time for calcul the mask position with numpy : 0.03208494186401367 nb_pixel_total : 45392 time to create 1 rle with old method : 0.04910731315612793 time for calcul the mask position with numpy : 0.03271937370300293 nb_pixel_total : 83768 time to create 1 rle with old method : 0.0918419361114502 time for calcul the mask position with numpy : 0.03345537185668945 nb_pixel_total : 57142 time to create 1 rle with old method : 0.06158280372619629 time for calcul the mask position with numpy : 0.03237485885620117 nb_pixel_total : 29394 time to create 1 rle with old method : 0.03165411949157715 time for calcul the mask position with numpy : 0.0327906608581543 nb_pixel_total : 123693 time to create 1 rle with old method : 0.1341075897216797 time for calcul the mask position with numpy : 0.032286643981933594 nb_pixel_total : 126835 time to create 1 rle with old method : 0.1349020004272461 time for calcul the mask position with numpy : 0.033109188079833984 nb_pixel_total : 4034 time to create 1 rle with old method : 0.0046460628509521484 time for calcul the mask position with numpy : 0.033377647399902344 nb_pixel_total : 75505 time to create 1 rle with old method : 0.08064532279968262 time for calcul the mask position with numpy : 0.03223133087158203 nb_pixel_total : 12208 time to create 1 rle with old method : 0.012996196746826172 time for calcul the mask position with numpy : 0.03177165985107422 nb_pixel_total : 6307 time to create 1 rle with old method : 0.006728649139404297 create new chi : 3.2872323989868164 time to delete rle : 0.00225830078125 batch 1 Loaded 55 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++Number RLEs to save : 20846 TO DO : save crop sub photo not yet done ! save time : 1.271681785583496 nb_obj : 17 nb_hashtags : 4 time to prepare the origin masks : 1.4532654285430908 time for calcul the mask position with numpy : 0.4995536804199219 nb_pixel_total : 7233144 time to create 1 rle with new method : 0.6300115585327148 time for calcul the mask position with numpy : 0.02240300178527832 nb_pixel_total : 29101 time to create 1 rle with old method : 0.03101205825805664 time for calcul the mask position with numpy : 0.023626089096069336 nb_pixel_total : 164342 time to create 1 rle with new method : 0.3237266540527344 time for calcul the mask position with numpy : 0.02316761016845703 nb_pixel_total : 74016 time to create 1 rle with old method : 0.07908844947814941 time for calcul the mask position with numpy : 0.024120807647705078 nb_pixel_total : 98809 time to create 1 rle with old method : 0.10857653617858887 time for calcul the mask position with numpy : 0.024141550064086914 nb_pixel_total : 8231 time to create 1 rle with old method : 0.00950002670288086 time for calcul the mask position with numpy : 0.02847003936767578 nb_pixel_total : 111184 time to create 1 rle with old method : 0.12508726119995117 time for calcul the mask position with numpy : 0.023395776748657227 nb_pixel_total : 90925 time to create 1 rle with old method : 0.09998631477355957 time for calcul the mask position with numpy : 0.023307323455810547 nb_pixel_total : 64725 time to create 1 rle with old method : 0.07037806510925293 time for calcul the mask position with numpy : 0.023385047912597656 nb_pixel_total : 38308 time to create 1 rle with old method : 0.04099583625793457 time for calcul the mask position with numpy : 0.022758960723876953 nb_pixel_total : 84460 time to create 1 rle with old method : 0.0895085334777832 time for calcul the mask position with numpy : 0.02448248863220215 nb_pixel_total : 144535 time to create 1 rle with old method : 0.15329694747924805 time for calcul the mask position with numpy : 0.025110244750976562 nb_pixel_total : 13410 time to create 1 rle with old method : 0.021377086639404297 time for calcul the mask position with numpy : 0.026210784912109375 nb_pixel_total : 14191 time to create 1 rle with old method : 0.02259206771850586 time for calcul the mask position with numpy : 0.02436089515686035 nb_pixel_total : 14382 time to create 1 rle with old method : 0.016716957092285156 time for calcul the mask position with numpy : 0.023166418075561523 nb_pixel_total : 14337 time to create 1 rle with old method : 0.016372203826904297 time for calcul the mask position with numpy : 0.02306985855102539 nb_pixel_total : 31957 time to create 1 rle with old method : 0.03410458564758301 time for calcul the mask position with numpy : 0.0232999324798584 nb_pixel_total : 64343 time to create 1 rle with old method : 0.06948471069335938 create new chi : 2.910426616668701 time to delete rle : 0.0016934871673583984 batch 1 Loaded 35 chid ids of type : 3594 ++++++++++++++++++Number RLEs to save : 12938 TO DO : save crop sub photo not yet done ! save time : 0.8094699382781982 nb_obj : 9 nb_hashtags : 3 time to prepare the origin masks : 0.4447190761566162 time for calcul the mask position with numpy : 0.062262773513793945 nb_pixel_total : 4780108 time to create 1 rle with new method : 0.15620827674865723 time for calcul the mask position with numpy : 0.023624420166015625 nb_pixel_total : 30048 time to create 1 rle with old method : 0.03313946723937988 time for calcul the mask position with numpy : 0.055652618408203125 nb_pixel_total : 3337406 time to create 1 rle with new method : 0.155470609664917 time for calcul the mask position with numpy : 0.023424625396728516 nb_pixel_total : 18228 time to create 1 rle with old method : 0.020378828048706055 time for calcul the mask position with numpy : 0.022697925567626953 nb_pixel_total : 45870 time to create 1 rle with old method : 0.04958462715148926 time for calcul the mask position with numpy : 0.02359938621520996 nb_pixel_total : 4320 time to create 1 rle with old method : 0.004802227020263672 time for calcul the mask position with numpy : 0.02307295799255371 nb_pixel_total : 35860 time to create 1 rle with old method : 0.038721323013305664 time for calcul the mask position with numpy : 0.023023128509521484 nb_pixel_total : 14881 time to create 1 rle with old method : 0.015509605407714844 time for calcul the mask position with numpy : 0.0227048397064209 nb_pixel_total : 14778 time to create 1 rle with old method : 0.01587080955505371 time for calcul the mask position with numpy : 0.02243804931640625 nb_pixel_total : 12901 time to create 1 rle with old method : 0.013463735580444336 create new chi : 0.8445754051208496 time to delete rle : 0.0012462139129638672 batch 1 Loaded 19 chid ids of type : 3594 +++++++++++Number RLEs to save : 9540 TO DO : save crop sub photo not yet done ! save time : 0.6893577575683594 map_output_result : {1362198073: (0.0, 'Should be the crop_list due to order', 0), 1362198068: (0.0, 'Should be the crop_list due to order', 0), 1362198066: (0.0, 'Should be the crop_list due to order', 0), 1362198064: (0.0, 'Should be the crop_list due to order', 0), 1362197939: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1362198073, 1362198068, 1362198066, 1362198064, 1362197939] Looping around the photos to save general results len do output : 5 /1362198073.Didn't retrieve data . /1362198068.Didn't retrieve data . /1362198066.Didn't retrieve data . /1362198064.Didn't retrieve data . /1362197939.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2994566') ('3318', '23497861', '1362198073', None, None, None, None, None, '2994566') ('3318', None, None, None, None, None, None, None, '2994566') ('3318', '23497861', '1362198068', None, None, None, None, None, '2994566') ('3318', None, None, None, None, None, None, None, '2994566') ('3318', '23497861', '1362198066', None, None, None, None, None, '2994566') ('3318', None, None, None, None, None, None, None, '2994566') ('3318', '23497861', '1362198064', None, None, None, None, None, '2994566') ('3318', None, None, None, None, None, None, None, '2994566') ('3318', '23497861', '1362197939', None, None, None, None, None, '2994566') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 15 time used for this insertion : 0.012682437896728516 save_final save missing photos in datou_result : time spend for datou_step_exec : 31.86521816253662 time spend to save output : 0.013049602508544922 total time spend for step 3 : 31.878267765045166 step4:ventilate_hashtags_in_portfolio Sat May 31 15:13:07 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 23497861 get user id for portfolio 23497861 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23497861 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pehd','papier','autre','environnement','pet_fonce','mal_croppe','metal','flou','pet_clair','background','carton')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23497861 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pehd','papier','autre','environnement','pet_fonce','mal_croppe','metal','flou','pet_clair','background','carton')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23497861 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pehd','papier','autre','environnement','pet_fonce','mal_croppe','metal','flou','pet_clair','background','carton')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/23498350,23498351,23498352,23498353,23498354,23498355,23498356,23498357,23498358,23498359,23498360?tags=pehd,papier,autre,environnement,pet_fonce,mal_croppe,metal,flou,pet_clair,background,carton Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1362198073, 1362198068, 1362198066, 1362198064, 1362197939] Looping around the photos to save general results len do output : 1 /23497861. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2994566') ('3318', '23497861', '1362198073', None, None, None, None, None, '2994566') ('3318', None, None, None, None, None, None, None, '2994566') ('3318', '23497861', '1362198068', None, None, None, None, None, '2994566') ('3318', None, None, None, None, None, None, None, '2994566') ('3318', '23497861', '1362198066', None, None, None, None, None, '2994566') ('3318', None, None, None, None, None, None, None, '2994566') ('3318', '23497861', '1362198064', None, None, None, None, None, '2994566') ('3318', None, None, None, None, None, None, None, '2994566') ('3318', '23497861', '1362197939', None, None, None, None, None, '2994566') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.01978135108947754 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.6308300495147705 time spend to save output : 0.020144224166870117 total time spend for step 4 : 1.6509742736816406 step5:final Sat May 31 15:13:08 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1362198073: ('0.21837635030864197',), 1362198068: ('0.21837635030864197',), 1362198066: ('0.21837635030864197',), 1362198064: ('0.21837635030864197',), 1362197939: ('0.21837635030864197',)} new output for save of step final : {1362198073: ('0.21837635030864197',), 1362198068: ('0.21837635030864197',), 1362198066: ('0.21837635030864197',), 1362198064: ('0.21837635030864197',), 1362197939: ('0.21837635030864197',)} [1362198073, 1362198068, 1362198066, 1362198064, 1362197939] Looping around the photos to save general results len do output : 5 /1362198073.Didn't retrieve data . /1362198068.Didn't retrieve data . /1362198066.Didn't retrieve data . /1362198064.Didn't retrieve data . /1362197939.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2994566') ('3318', '23497861', '1362198073', None, None, None, None, None, '2994566') ('3318', None, None, None, None, None, None, None, '2994566') ('3318', '23497861', '1362198068', None, None, None, None, None, '2994566') ('3318', None, None, None, None, None, None, None, '2994566') ('3318', '23497861', '1362198066', None, None, None, None, None, '2994566') ('3318', None, None, None, None, None, None, None, '2994566') ('3318', '23497861', '1362198064', None, None, None, None, None, '2994566') ('3318', None, None, None, None, None, None, None, '2994566') ('3318', '23497861', '1362197939', None, None, None, None, None, '2994566') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 15 time used for this insertion : 0.01250457763671875 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.11295890808105469 time spend to save output : 0.012792110443115234 total time spend for step 5 : 0.12575101852416992 step6:blur_detection Sat May 31 15:13:08 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939.jpg resize: (2160, 3840) 1362198073 -6.347928691283052 treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7.jpg resize: (2160, 3840) 1362198068 -6.321373549024502 treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa.jpg resize: (2160, 3840) 1362198066 -5.8081235508788245 treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328.jpg resize: (2160, 3840) 1362198064 -6.281739732233359 treat image : temp/1748697028_612518_1362197939_bd7dbae194618e361cf677542e299592.jpg resize: (2160, 3840) 1362197939 -3.370428542583224 treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383805_0.png resize: (171, 136) 1362226449 -2.866994439202667 treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383798_0.png resize: (306, 151) 1362226450 -4.191227029023158 treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383799_0.png resize: (218, 155) 1362226451 -0.9844106550292351 treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383790_0.png resize: (437, 373) 1362226452 -5.027503127032915 treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383801_0.png resize: (198, 345) 1362226453 -1.265423273319571 treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383785_0.png resize: (174, 179) 1362226454 -1.0278207125994527 treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383789_0.png resize: (275, 293) 1362226455 -3.0582229586197207 treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383788_0.png resize: (198, 150) 1362226456 -3.351960459777534 treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383800_0.png resize: (438, 363) 1362226457 -3.019767101322815 treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383802_0.png resize: (99, 270) 1362226458 -0.21390091760786015 treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383791_0.png resize: (97, 95) 1362226459 -3.5120162265863204 treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383783_0.png resize: (270, 280) 1362226460 -4.058120658406571 treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383787_0.png resize: (568, 331) 1362226461 -2.722240206252081 treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383786_0.png resize: (317, 333) 1362226462 2.8263614215847257 treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383792_0.png resize: (193, 378) 1362226463 -3.8305299133869872 treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383793_0.png resize: (251, 131) 1362226464 -2.9537228082256566 treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383794_0.png resize: (218, 193) 1362226465 -1.4231751249651776 treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383795_0.png resize: (164, 155) 1362226466 -3.462806097211596 treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383796_0.png resize: (582, 268) 1362226467 -3.7138465672523555 treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383828_0.png resize: (536, 216) 1362226468 -2.590661093790119 treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383827_0.png resize: (322, 264) 1362226469 -2.2362256108704313 treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383810_0.png resize: (362, 311) 1362226470 0.5363247090680082 treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383817_0.png resize: (184, 95) 1362226471 -2.135044334173491 treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383818_0.png resize: (205, 350) 1362226472 -3.3990625096726097 treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383819_0.png resize: (477, 409) 1362226473 -1.5641270312764686 treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383821_0.png resize: (91, 122) 1362226474 -1.9736419874893596 treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383824_0.png resize: (258, 386) 1362226475 -3.9351506169375776 treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383814_0.png resize: (324, 242) 1362226476 -3.887115860843582 treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383813_0.png resize: (538, 416) 1362226477 -1.1109878225348628 treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383811_0.png resize: (216, 194) 1362226478 -1.0406899422855718 treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383809_0.png resize: (659, 419) 1362226479 -4.021783423321063 treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383816_0.png resize: (190, 301) 1362226480 -2.2476350236235283 treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383820_0.png resize: (88, 94) 1362226481 -2.919429002687161 treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383812_0.png resize: (220, 117) 1362226482 -4.4066012556376695 treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383815_0.png resize: (68, 184) 1362226483 -3.4187543498893493 treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383826_0.png resize: (208, 214) 1362226484 -3.770528617200868 treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383839_0.png resize: (606, 217) 1362226485 -3.157060999943625 treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383837_0.png resize: (211, 141) 1362226486 -1.8048830960125344 treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383852_0.png resize: (259, 156) 1362226487 -4.093182769554704 treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383834_0.png resize: (486, 541) 1362226488 -3.295936270929765 treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383843_0.png resize: (119, 135) 1362226490 -0.6769040297104042 treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383846_0.png resize: (95, 74) 1362226491 -2.817239515705038 treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383829_0.png resize: (382, 856) 1362226492 -2.023770318470798 treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383841_0.png resize: (113, 118) 1362226494 -1.3158988275457026 treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383835_0.png resize: (458, 282) 1362226495 -4.307775992598028 treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383854_0.png resize: (663, 532) 1362226496 -3.4468548804954713 treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383851_0.png resize: (333, 483) 1362226498 -1.4425189529745048 treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383842_0.png resize: (266, 386) 1362226499 -2.243043081127041 treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383853_0.png resize: (161, 171) 1362226500 -2.559254171132688 treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383830_0.png resize: (209, 252) 1362226502 -3.9341655618958873 treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383836_0.png resize: (195, 208) 1362226503 -0.5650838524109084 treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383845_0.png resize: (84, 126) 1362226504 -3.088257406427085 treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383838_0.png resize: (762, 226) 1362226506 -1.3175130188615032 treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383840_0.png resize: (209, 278) 1362226507 -2.1674956815992434 treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383871_0.png resize: (507, 506) 1362226508 -4.07862692369101 treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383857_0.png resize: (207, 264) 1362226510 -0.31639831679381497 treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383868_0.png resize: (89, 118) 1362226511 -2.2993855490175537 treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383861_0.png resize: (147, 149) 1362226513 -1.9650966917010981 treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383856_0.png resize: (241, 400) 1362226515 -3.6745968138148375 treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383860_0.png resize: (191, 97) 1362226516 -2.4519246475305048 treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383870_0.png resize: (302, 337) 1362226518 -3.5613986865223803 treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383859_0.png resize: (107, 170) 1362226520 -3.4583921815524703 treat image : temp/1748697028_612518_1362197939_bd7dbae194618e361cf677542e299592_rle_crop_3818383876_0.png resize: (245, 244) 1362226521 -4.314552897132235 treat image : temp/1748697028_612518_1362197939_bd7dbae194618e361cf677542e299592_rle_crop_3818383873_0.png resize: (127, 137) 1362226522 -3.944005612357274 treat image : temp/1748697028_612518_1362197939_bd7dbae194618e361cf677542e299592_rle_crop_3818383875_0.png resize: (159, 142) 1362226524 -3.722686728366123 treat image : temp/1748697028_612518_1362197939_bd7dbae194618e361cf677542e299592_rle_crop_3818383879_0.png resize: (153, 160) 1362226525 -4.4139578133137185 treat image : temp/1748697028_612518_1362197939_bd7dbae194618e361cf677542e299592_rle_crop_3818383877_0.png resize: (122, 108) 1362226526 -3.336736880810761 treat image : temp/1748697028_612518_1362197939_bd7dbae194618e361cf677542e299592_rle_crop_3818383874_0.png resize: (190, 141) 1362226527 -3.759489035942662 treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383784_0.png resize: (106, 172) 1362226543 -3.0764810439724504 treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383807_0.png resize: (426, 182) 1362226544 -2.7317716690713767 treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383806_0.png resize: (161, 65) 1362226546 -2.9375544860075755 treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383822_0.png resize: (170, 63) 1362226547 -2.431441199262047 treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383858_0.png resize: (157, 167) 1362226548 -3.9975583361426468 treat image : temp/1748697028_612518_1362197939_bd7dbae194618e361cf677542e299592_rle_crop_3818383878_0.png resize: (230, 238) 1362226560 -3.4979381757065138 treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383803_0.png resize: (450, 365) 1362226853 -3.5385346153515678 treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383797_0.png resize: (284, 361) 1362226855 -4.474255816754582 treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383808_0.png resize: (324, 288) 1362226856 -5.27519210139002 treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383823_0.png resize: (246, 297) 1362226858 -2.6939652274398362 treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383833_0.png resize: (383, 478) 1362226860 -4.408930399710065 treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383849_0.png resize: (444, 612) 1362226861 -3.375182769254567 treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383832_0.png resize: (378, 450) 1362226863 -4.899306334972415 treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383847_0.png resize: (354, 264) 1362226866 -3.2186279732451584 treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383831_0.png resize: (395, 375) 1362226867 -3.4747434372579056 treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383850_0.png resize: (400, 575) 1362226869 -3.315803653531441 treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383848_0.png resize: (362, 326) 1362226871 -2.4022928505599097 treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383844_0.png resize: (404, 186) 1362226872 -4.771185003398474 treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383866_0.png resize: (538, 229) 1362226874 -4.943576197441293 treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383867_0.png resize: (425, 340) 1362226875 -2.322544331921455 treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383863_0.png resize: (431, 332) 1362226876 -3.729505739119253 treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383862_0.png resize: (471, 462) 1362226879 -4.418956088970036 treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383872_0.png resize: (110, 445) 1362226880 -3.9705044118096953 treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383869_0.png resize: (325, 369) 1362226881 -4.187110192418784 treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383864_0.png resize: (267, 181) 1362226883 -3.544253718200673 treat image : temp/1748697028_612518_1362197939_bd7dbae194618e361cf677542e299592_rle_crop_3818383880_0.png resize: (1700, 2804) 1362226884 -3.6730472776880774 treat image : temp/1748697028_612518_1362197939_bd7dbae194618e361cf677542e299592_rle_crop_3818383881_0.png resize: (274, 355) 1362226885 -4.6004595203035015 treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383804_0.png resize: (279, 293) 1362226900 -2.8971137656217647 treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383825_0.png resize: (259, 229) 1362226901 -4.2119122922385435 treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383855_0.png resize: (152, 370) 1362226903 -3.5195672924256383 treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383865_0.png resize: (310, 301) 1362226951 -3.6883631643664123 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 104 time used for this insertion : 0.020963430404663086 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 104 time used for this insertion : 0.03815293312072754 save missing photos in datou_result : time spend for datou_step_exec : 20.29960036277771 time spend to save output : 0.06400275230407715 total time spend for step 6 : 20.363603115081787 step7:brightness Sat May 31 15:13:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939.jpg treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7.jpg treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa.jpg treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328.jpg treat image : temp/1748697028_612518_1362197939_bd7dbae194618e361cf677542e299592.jpg treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383805_0.png treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383798_0.png treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383799_0.png treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383790_0.png treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383801_0.png treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383785_0.png treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383789_0.png treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383788_0.png treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383800_0.png treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383802_0.png treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383791_0.png treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383783_0.png treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383787_0.png treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383786_0.png treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383792_0.png treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383793_0.png treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383794_0.png treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383795_0.png treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383796_0.png treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383828_0.png treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383827_0.png treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383810_0.png treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383817_0.png treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383818_0.png treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383819_0.png treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383821_0.png treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383824_0.png treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383814_0.png treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383813_0.png treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383811_0.png treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383809_0.png treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383816_0.png treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383820_0.png treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383812_0.png treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383815_0.png treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383826_0.png treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383839_0.png treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383837_0.png treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383852_0.png treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383834_0.png treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383843_0.png treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383846_0.png treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383829_0.png treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383841_0.png treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383835_0.png treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383854_0.png treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383851_0.png treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383842_0.png treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383853_0.png treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383830_0.png treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383836_0.png treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383845_0.png treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383838_0.png treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383840_0.png treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383871_0.png treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383857_0.png treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383868_0.png treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383861_0.png treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383856_0.png treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383860_0.png treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383870_0.png treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383859_0.png treat image : temp/1748697028_612518_1362197939_bd7dbae194618e361cf677542e299592_rle_crop_3818383876_0.png treat image : temp/1748697028_612518_1362197939_bd7dbae194618e361cf677542e299592_rle_crop_3818383873_0.png treat image : temp/1748697028_612518_1362197939_bd7dbae194618e361cf677542e299592_rle_crop_3818383875_0.png treat image : temp/1748697028_612518_1362197939_bd7dbae194618e361cf677542e299592_rle_crop_3818383879_0.png treat image : temp/1748697028_612518_1362197939_bd7dbae194618e361cf677542e299592_rle_crop_3818383877_0.png treat image : temp/1748697028_612518_1362197939_bd7dbae194618e361cf677542e299592_rle_crop_3818383874_0.png treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383784_0.png treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383807_0.png treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383806_0.png treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383822_0.png treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383858_0.png treat image : temp/1748697028_612518_1362197939_bd7dbae194618e361cf677542e299592_rle_crop_3818383878_0.png treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383803_0.png treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383797_0.png treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383808_0.png treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383823_0.png treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383833_0.png treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383849_0.png treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383832_0.png treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383847_0.png treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383831_0.png treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383850_0.png treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383848_0.png treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383844_0.png treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383866_0.png treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383867_0.png treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383863_0.png treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383862_0.png treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383872_0.png treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383869_0.png treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383864_0.png treat image : temp/1748697028_612518_1362197939_bd7dbae194618e361cf677542e299592_rle_crop_3818383880_0.png treat image : temp/1748697028_612518_1362197939_bd7dbae194618e361cf677542e299592_rle_crop_3818383881_0.png treat image : temp/1748697028_612518_1362198073_2252b9aa0118a37c673084745f7c0939_rle_crop_3818383804_0.png treat image : temp/1748697028_612518_1362198068_a228a355575bae77ed11e8600ffbc7d7_rle_crop_3818383825_0.png treat image : temp/1748697028_612518_1362198066_1dc9685cb30405d3d7fdf63f679a8faa_rle_crop_3818383855_0.png treat image : temp/1748697028_612518_1362198064_0581a033abe96340fe561625b3a12328_rle_crop_3818383865_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 104 time used for this insertion : 0.017915964126586914 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 104 time used for this insertion : 0.029233694076538086 save missing photos in datou_result : time spend for datou_step_exec : 6.014717102050781 time spend to save output : 0.05219459533691406 total time spend for step 7 : 6.066911697387695 step8:velours_tree Sat May 31 15:13:35 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.0878145694732666 time spend to save output : 3.933906555175781e-05 total time spend for step 8 : 0.08785390853881836 step9:send_mail_cod Sat May 31 15:13:35 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P23497861_31-05-2025_15_13_35.pdf 23498350 imagette234983501748697215 23498351 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette234983511748697215 23498352 change filename to text .change filename to text .change filename to text .imagette234983521748697216 23498354 change filename to text .imagette234983541748697217 23498355 imagette234983551748697217 23498356 change filename to text .imagette234983561748697217 23498357 imagette234983571748697217 23498358 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette234983581748697217 23498359 imagette234983591748697219 23498360 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette234983601748697219 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=23497861 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/23498350,23498351,23498352,23498353,23498354,23498355,23498356,23498357,23498358,23498359,23498360?tags=pehd,papier,autre,environnement,pet_fonce,mal_croppe,metal,flou,pet_clair,background,carton args[1362198073] : ((1362198073, -6.347928691283052, 492609224), (1362198073, 0.7433380758007985, 2107752395), '0.21837635030864197') We are sending mail with results at report@fotonower.com args[1362198068] : ((1362198068, -6.321373549024502, 492609224), (1362198068, 0.6441228429462029, 2107752395), '0.21837635030864197') We are sending mail with results at report@fotonower.com args[1362198066] : ((1362198066, -5.8081235508788245, 492609224), (1362198066, 0.5323829376742595, 2107752395), '0.21837635030864197') We are sending mail with results at report@fotonower.com args[1362198064] : ((1362198064, -6.281739732233359, 492609224), (1362198064, 0.5561852212550853, 2107752395), '0.21837635030864197') We are sending mail with results at report@fotonower.com args[1362197939] : ((1362197939, -3.370428542583224, 492609224), (1362197939, -1.3328256131817147, 501862349), '0.21837635030864197') We are sending mail with results at report@fotonower.com refus_total : 0.21837635030864197 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=23497861 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23497861_31-05-2025_15_13_35.pdf results_Auto_P23497861_31-05-2025_15_13_35.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23497861_31-05-2025_15_13_35.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','23497861','results_Auto_P23497861_31-05-2025_15_13_35.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23497861_31-05-2025_15_13_35.pdf','pdf','','0.99','0.21837635030864197') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/23497861

https://www.fotonower.com/image?json=false&list_photos_id=1362198073
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1362198068
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1362198066
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1362198064
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1362197939
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 21.84%
Veuillez trouver les photos des contaminants.

exemples de contaminants: papier: https://www.fotonower.com/view/23498351?limit=200
exemples de contaminants: autre: https://www.fotonower.com/view/23498352?limit=200
exemples de contaminants: pet_fonce: https://www.fotonower.com/view/23498354?limit=200
exemples de contaminants: metal: https://www.fotonower.com/view/23498356?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/23498358?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/23498360?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23497861_31-05-2025_15_13_35.pdf.

Lien vers velours :https://www.fotonower.com/velours/23498350,23498351,23498352,23498353,23498354,23498355,23498356,23498357,23498358,23498359,23498360?tags=pehd,papier,autre,environnement,pet_fonce,mal_croppe,metal,flou,pet_clair,background,carton.


L'équipe Fotonower 202 b'' Server: nginx Date: Sat, 31 May 2025 13:13:42 GMT Content-Length: 0 Connection: close X-Message-Id: vGiLV_oITQ2EehYV8BBH9g Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1362198073, 1362198068, 1362198066, 1362198064, 1362197939] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2994566') ('3318', '23497861', '1362198073', None, None, None, None, None, '2994566') ('3318', None, None, None, None, None, None, None, '2994566') ('3318', '23497861', '1362198068', None, None, None, None, None, '2994566') ('3318', None, None, None, None, None, None, None, '2994566') ('3318', '23497861', '1362198066', None, None, None, None, None, '2994566') ('3318', None, None, None, None, None, None, None, '2994566') ('3318', '23497861', '1362198064', None, None, None, None, None, '2994566') ('3318', None, None, None, None, None, None, None, '2994566') ('3318', '23497861', '1362197939', None, None, None, None, None, '2994566') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 5 time used for this insertion : 0.013109683990478516 save_final save missing photos in datou_result : time spend for datou_step_exec : 6.974676847457886 time spend to save output : 0.01329183578491211 total time spend for step 9 : 6.987968683242798 step10:split_time_score Sat May 31 15:13:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('13', 5),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 31052025 23497861 Nombre de photos uploadées : 5 / 23040 (0%) 31052025 23497861 Nombre de photos taguées (types de déchets): 0 / 5 (0%) 31052025 23497861 Nombre de photos taguées (volume) : 0 / 5 (0%) elapsed_time : load_data_split_time_score 1.6689300537109375e-06 elapsed_time : order_list_meta_photo_and_scores 5.0067901611328125e-06 ????? elapsed_time : fill_and_build_computed_from_old_data 0.00032711029052734375 elapsed_time : insert_dashboard_record_day_entry 0.028461217880249023 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.14137616946373457 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23494767_31-05-2025_13_55_18.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23494767 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23494767 AND mptpi.`type`=3594 To do Qualite : 0.1274980490670595 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23494769_31-05-2025_13_46_52.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23494769 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23494769 AND mptpi.`type`=3594 To do Qualite : 0.21837635030864197 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23497861_31-05-2025_15_13_35.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23497861 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23497861 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'31052025': {'nb_upload': 5, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1362198073, 1362198068, 1362198066, 1362198064, 1362197939] Looping around the photos to save general results len do output : 1 /23497861Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2994566') ('3318', '23497861', '1362198073', None, None, None, None, None, '2994566') ('3318', None, None, None, None, None, None, None, '2994566') ('3318', '23497861', '1362198068', None, None, None, None, None, '2994566') ('3318', None, None, None, None, None, None, None, '2994566') ('3318', '23497861', '1362198066', None, None, None, None, None, '2994566') ('3318', None, None, None, None, None, None, None, '2994566') ('3318', '23497861', '1362198064', None, None, None, None, None, '2994566') ('3318', None, None, None, None, None, None, None, '2994566') ('3318', '23497861', '1362197939', None, None, None, None, None, '2994566') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.015556812286376953 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.3917655944824219 time spend to save output : 0.01575160026550293 total time spend for step 10 : 0.4075171947479248 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 5 set_done_treatment 122.71user 37.72system 3:18.59elapsed 80%CPU (0avgtext+0avgdata 4827296maxresident)k 501000inputs+146200outputs (18major+7393462minor)pagefaults 0swaps