python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 1944000 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['2739209'] with mtr_portfolio_ids : ['22230710'] and first list_photo_ids : [] new path : /proc/1944000/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 8 ; length of list_pids : 8 ; length of list_args : 8 time to download the photos : 1.2556993961334229 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Sat Apr 19 01:30:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10814 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-04-19 01:30:33.170922: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-04-19 01:30:33.179056: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-04-19 01:30:33.180501: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f0f58000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-04-19 01:30:33.180521: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-04-19 01:30:33.183282: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-04-19 01:30:33.306954: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x3b96e840 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-04-19 01:30:33.306995: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-04-19 01:30:33.307944: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-04-19 01:30:33.308262: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-04-19 01:30:33.310218: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-04-19 01:30:33.312240: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-04-19 01:30:33.312551: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-04-19 01:30:33.314753: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-04-19 01:30:33.315840: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-04-19 01:30:33.320108: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-04-19 01:30:33.321581: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-04-19 01:30:33.321677: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-04-19 01:30:33.322472: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-04-19 01:30:33.322495: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-04-19 01:30:33.322504: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-04-19 01:30:33.324033: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9879 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-04-19 01:30:33.580205: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-04-19 01:30:33.580331: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-04-19 01:30:33.580351: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-04-19 01:30:33.580369: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-04-19 01:30:33.580386: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-04-19 01:30:33.580404: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-04-19 01:30:33.580420: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-04-19 01:30:33.580438: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-04-19 01:30:33.581623: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-04-19 01:30:33.582799: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-04-19 01:30:33.582831: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-04-19 01:30:33.582846: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-04-19 01:30:33.582860: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-04-19 01:30:33.582874: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-04-19 01:30:33.582888: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-04-19 01:30:33.582919: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-04-19 01:30:33.582937: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-04-19 01:30:33.584190: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-04-19 01:30:33.584224: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-04-19 01:30:33.584233: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-04-19 01:30:33.584241: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-04-19 01:30:33.585478: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9879 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-04-19 01:30:42.101960: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-04-19 01:30:42.284388: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 8 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 11.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 18 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 11.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 12 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 27 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 2.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 37 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 48 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 31 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 4.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 22 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 15 Detection mask done ! Trying to reset tf kernel 1944694 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 555 tf kernel not reseted sub process len(results) : 8 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 8 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5844 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0007283687591552734 nb_pixel_total : 16183 time to create 1 rle with old method : 0.02118992805480957 length of segment : 119 time for calcul the mask position with numpy : 0.26026129722595215 nb_pixel_total : 692546 time to create 1 rle with new method : 0.06731700897216797 length of segment : 498 time for calcul the mask position with numpy : 0.0007078647613525391 nb_pixel_total : 41787 time to create 1 rle with old method : 0.05082988739013672 length of segment : 332 time for calcul the mask position with numpy : 0.0080108642578125 nb_pixel_total : 382449 time to create 1 rle with new method : 0.041580915451049805 length of segment : 665 time for calcul the mask position with numpy : 0.002398967742919922 nb_pixel_total : 100733 time to create 1 rle with old method : 0.12334966659545898 length of segment : 229 time for calcul the mask position with numpy : 0.0008683204650878906 nb_pixel_total : 39624 time to create 1 rle with old method : 0.05074453353881836 length of segment : 300 time for calcul the mask position with numpy : 0.018500804901123047 nb_pixel_total : 535412 time to create 1 rle with new method : 0.05815005302429199 length of segment : 828 time for calcul the mask position with numpy : 0.0006577968597412109 nb_pixel_total : 30612 time to create 1 rle with old method : 0.038452863693237305 length of segment : 171 time for calcul the mask position with numpy : 0.00024509429931640625 nb_pixel_total : 8522 time to create 1 rle with old method : 0.010889530181884766 length of segment : 149 time for calcul the mask position with numpy : 0.0153656005859375 nb_pixel_total : 125246 time to create 1 rle with old method : 0.15729522705078125 length of segment : 519 time for calcul the mask position with numpy : 0.0005917549133300781 nb_pixel_total : 33320 time to create 1 rle with old method : 0.03949570655822754 length of segment : 255 time for calcul the mask position with numpy : 0.0013093948364257812 nb_pixel_total : 101602 time to create 1 rle with old method : 0.1260526180267334 length of segment : 312 time for calcul the mask position with numpy : 0.0019519329071044922 nb_pixel_total : 73436 time to create 1 rle with old method : 0.08535122871398926 length of segment : 354 time for calcul the mask position with numpy : 0.006115913391113281 nb_pixel_total : 260935 time to create 1 rle with new method : 0.02272486686706543 length of segment : 1033 time for calcul the mask position with numpy : 0.007761955261230469 nb_pixel_total : 334707 time to create 1 rle with new method : 0.09390926361083984 length of segment : 828 time for calcul the mask position with numpy : 0.004643917083740234 nb_pixel_total : 149912 time to create 1 rle with old method : 0.1806337833404541 length of segment : 789 time for calcul the mask position with numpy : 0.00048804283142089844 nb_pixel_total : 8492 time to create 1 rle with old method : 0.010402917861938477 length of segment : 111 time for calcul the mask position with numpy : 0.0015103816986083984 nb_pixel_total : 36418 time to create 1 rle with old method : 0.04545950889587402 length of segment : 221 time for calcul the mask position with numpy : 0.001699209213256836 nb_pixel_total : 32192 time to create 1 rle with old method : 0.03975939750671387 length of segment : 291 time for calcul the mask position with numpy : 0.0005633831024169922 nb_pixel_total : 7527 time to create 1 rle with old method : 0.00896453857421875 length of segment : 109 time for calcul the mask position with numpy : 0.0007066726684570312 nb_pixel_total : 9676 time to create 1 rle with old method : 0.016014814376831055 length of segment : 126 time for calcul the mask position with numpy : 0.0007238388061523438 nb_pixel_total : 13060 time to create 1 rle with old method : 0.021265029907226562 length of segment : 176 time for calcul the mask position with numpy : 0.0005707740783691406 nb_pixel_total : 14585 time to create 1 rle with old method : 0.020061492919921875 length of segment : 135 time for calcul the mask position with numpy : 0.0010750293731689453 nb_pixel_total : 19167 time to create 1 rle with old method : 0.022075891494750977 length of segment : 197 time for calcul the mask position with numpy : 0.0016002655029296875 nb_pixel_total : 35995 time to create 1 rle with old method : 0.04094243049621582 length of segment : 284 time for calcul the mask position with numpy : 0.0004239082336425781 nb_pixel_total : 7877 time to create 1 rle with old method : 0.009139299392700195 length of segment : 96 time for calcul the mask position with numpy : 0.0010275840759277344 nb_pixel_total : 23236 time to create 1 rle with old method : 0.028250694274902344 length of segment : 242 time for calcul the mask position with numpy : 0.0015888214111328125 nb_pixel_total : 43370 time to create 1 rle with old method : 0.05153083801269531 length of segment : 250 time for calcul the mask position with numpy : 0.0008599758148193359 nb_pixel_total : 12876 time to create 1 rle with old method : 0.016062021255493164 length of segment : 206 time for calcul the mask position with numpy : 0.0005841255187988281 nb_pixel_total : 13623 time to create 1 rle with old method : 0.017798662185668945 length of segment : 191 time for calcul the mask position with numpy : 0.0007612705230712891 nb_pixel_total : 21837 time to create 1 rle with old method : 0.026675939559936523 length of segment : 219 time for calcul the mask position with numpy : 0.0005204677581787109 nb_pixel_total : 8700 time to create 1 rle with old method : 0.010296344757080078 length of segment : 145 time for calcul the mask position with numpy : 0.0008821487426757812 nb_pixel_total : 17641 time to create 1 rle with old method : 0.02036309242248535 length of segment : 204 time for calcul the mask position with numpy : 0.001850128173828125 nb_pixel_total : 47083 time to create 1 rle with old method : 0.057698726654052734 length of segment : 248 time for calcul the mask position with numpy : 0.0012922286987304688 nb_pixel_total : 29834 time to create 1 rle with old method : 0.03665304183959961 length of segment : 190 time for calcul the mask position with numpy : 0.0006821155548095703 nb_pixel_total : 12583 time to create 1 rle with old method : 0.014454841613769531 length of segment : 186 time for calcul the mask position with numpy : 0.0014805793762207031 nb_pixel_total : 36907 time to create 1 rle with old method : 0.04460930824279785 length of segment : 166 time for calcul the mask position with numpy : 0.000560760498046875 nb_pixel_total : 9951 time to create 1 rle with old method : 0.011597394943237305 length of segment : 111 time for calcul the mask position with numpy : 0.0010883808135986328 nb_pixel_total : 18070 time to create 1 rle with old method : 0.020794391632080078 length of segment : 183 time for calcul the mask position with numpy : 0.00047397613525390625 nb_pixel_total : 8157 time to create 1 rle with old method : 0.009402990341186523 length of segment : 120 time for calcul the mask position with numpy : 0.0023648738861083984 nb_pixel_total : 41742 time to create 1 rle with old method : 0.04786801338195801 length of segment : 320 time for calcul the mask position with numpy : 0.0009095668792724609 nb_pixel_total : 18338 time to create 1 rle with old method : 0.021091461181640625 length of segment : 195 time for calcul the mask position with numpy : 0.0014421939849853516 nb_pixel_total : 27411 time to create 1 rle with old method : 0.03316998481750488 length of segment : 180 time for calcul the mask position with numpy : 0.0011761188507080078 nb_pixel_total : 14965 time to create 1 rle with old method : 0.01739645004272461 length of segment : 285 time for calcul the mask position with numpy : 0.0005300045013427734 nb_pixel_total : 9460 time to create 1 rle with old method : 0.010938644409179688 length of segment : 120 time for calcul the mask position with numpy : 0.0027589797973632812 nb_pixel_total : 47892 time to create 1 rle with old method : 0.054251670837402344 length of segment : 462 time for calcul the mask position with numpy : 0.0007531642913818359 nb_pixel_total : 13522 time to create 1 rle with old method : 0.015914440155029297 length of segment : 175 time for calcul the mask position with numpy : 0.0003745555877685547 nb_pixel_total : 8503 time to create 1 rle with old method : 0.010149478912353516 length of segment : 110 time for calcul the mask position with numpy : 0.0009739398956298828 nb_pixel_total : 14250 time to create 1 rle with old method : 0.01839160919189453 length of segment : 235 time for calcul the mask position with numpy : 0.0002541542053222656 nb_pixel_total : 3429 time to create 1 rle with old method : 0.004113435745239258 length of segment : 54 time for calcul the mask position with numpy : 0.0003254413604736328 nb_pixel_total : 5452 time to create 1 rle with old method : 0.0064809322357177734 length of segment : 91 time for calcul the mask position with numpy : 0.0029647350311279297 nb_pixel_total : 40332 time to create 1 rle with old method : 0.050316810607910156 length of segment : 727 time for calcul the mask position with numpy : 0.0004680156707763672 nb_pixel_total : 9854 time to create 1 rle with old method : 0.01219487190246582 length of segment : 186 time for calcul the mask position with numpy : 0.0010523796081542969 nb_pixel_total : 16556 time to create 1 rle with old method : 0.021129369735717773 length of segment : 146 time for calcul the mask position with numpy : 0.00608372688293457 nb_pixel_total : 92727 time to create 1 rle with old method : 0.11283087730407715 length of segment : 512 time for calcul the mask position with numpy : 0.00048613548278808594 nb_pixel_total : 9746 time to create 1 rle with old method : 0.012259721755981445 length of segment : 96 time for calcul the mask position with numpy : 0.002315998077392578 nb_pixel_total : 56756 time to create 1 rle with old method : 0.08565044403076172 length of segment : 285 time for calcul the mask position with numpy : 0.002124786376953125 nb_pixel_total : 32842 time to create 1 rle with old method : 0.04239916801452637 length of segment : 225 time for calcul the mask position with numpy : 0.012064695358276367 nb_pixel_total : 158306 time to create 1 rle with new method : 0.01923656463623047 length of segment : 677 time for calcul the mask position with numpy : 0.0012047290802001953 nb_pixel_total : 17244 time to create 1 rle with old method : 0.019701480865478516 length of segment : 200 time for calcul the mask position with numpy : 0.0033516883850097656 nb_pixel_total : 48994 time to create 1 rle with old method : 0.05883193016052246 length of segment : 514 time for calcul the mask position with numpy : 0.0034999847412109375 nb_pixel_total : 78666 time to create 1 rle with old method : 0.09527969360351562 length of segment : 345 time for calcul the mask position with numpy : 0.0018856525421142578 nb_pixel_total : 28535 time to create 1 rle with old method : 0.03507065773010254 length of segment : 265 time for calcul the mask position with numpy : 0.015040874481201172 nb_pixel_total : 296805 time to create 1 rle with new method : 0.026885509490966797 length of segment : 1074 time for calcul the mask position with numpy : 0.001603841781616211 nb_pixel_total : 24899 time to create 1 rle with old method : 0.030974864959716797 length of segment : 285 time for calcul the mask position with numpy : 0.0013384819030761719 nb_pixel_total : 14631 time to create 1 rle with old method : 0.017949581146240234 length of segment : 203 time for calcul the mask position with numpy : 0.0005116462707519531 nb_pixel_total : 6236 time to create 1 rle with old method : 0.00797891616821289 length of segment : 109 time for calcul the mask position with numpy : 0.0009396076202392578 nb_pixel_total : 25282 time to create 1 rle with old method : 0.03035569190979004 length of segment : 236 time for calcul the mask position with numpy : 0.0016963481903076172 nb_pixel_total : 44047 time to create 1 rle with old method : 0.053360700607299805 length of segment : 271 time for calcul the mask position with numpy : 0.0008451938629150391 nb_pixel_total : 14690 time to create 1 rle with old method : 0.016821861267089844 length of segment : 136 time for calcul the mask position with numpy : 0.004609584808349609 nb_pixel_total : 81505 time to create 1 rle with old method : 0.10204696655273438 length of segment : 339 time for calcul the mask position with numpy : 0.0009322166442871094 nb_pixel_total : 11130 time to create 1 rle with old method : 0.013843297958374023 length of segment : 119 time for calcul the mask position with numpy : 0.0034475326538085938 nb_pixel_total : 58194 time to create 1 rle with old method : 0.06781411170959473 length of segment : 308 time for calcul the mask position with numpy : 0.0011513233184814453 nb_pixel_total : 18972 time to create 1 rle with old method : 0.022681713104248047 length of segment : 202 time for calcul the mask position with numpy : 0.002263307571411133 nb_pixel_total : 64788 time to create 1 rle with old method : 0.07323884963989258 length of segment : 363 time for calcul the mask position with numpy : 0.004115104675292969 nb_pixel_total : 78421 time to create 1 rle with old method : 0.09458470344543457 length of segment : 282 time for calcul the mask position with numpy : 0.0011565685272216797 nb_pixel_total : 17834 time to create 1 rle with old method : 0.022361278533935547 length of segment : 206 time for calcul the mask position with numpy : 0.0018782615661621094 nb_pixel_total : 25052 time to create 1 rle with old method : 0.03154325485229492 length of segment : 246 time for calcul the mask position with numpy : 0.001565694808959961 nb_pixel_total : 44468 time to create 1 rle with old method : 0.05417966842651367 length of segment : 213 time for calcul the mask position with numpy : 0.0020456314086914062 nb_pixel_total : 36400 time to create 1 rle with old method : 0.04572010040283203 length of segment : 384 time for calcul the mask position with numpy : 0.0035390853881835938 nb_pixel_total : 84522 time to create 1 rle with old method : 0.10331988334655762 length of segment : 423 time for calcul the mask position with numpy : 0.0036199092864990234 nb_pixel_total : 77648 time to create 1 rle with old method : 0.11720943450927734 length of segment : 370 time for calcul the mask position with numpy : 0.003351926803588867 nb_pixel_total : 128470 time to create 1 rle with old method : 0.154937744140625 length of segment : 273 time for calcul the mask position with numpy : 0.002178668975830078 nb_pixel_total : 55590 time to create 1 rle with old method : 0.0688939094543457 length of segment : 371 time for calcul the mask position with numpy : 0.002736806869506836 nb_pixel_total : 53434 time to create 1 rle with old method : 0.06744551658630371 length of segment : 400 time for calcul the mask position with numpy : 0.017785072326660156 nb_pixel_total : 484003 time to create 1 rle with new method : 0.03940868377685547 length of segment : 933 time for calcul the mask position with numpy : 0.0006201267242431641 nb_pixel_total : 15998 time to create 1 rle with old method : 0.018537282943725586 length of segment : 102 time for calcul the mask position with numpy : 0.0007541179656982422 nb_pixel_total : 16197 time to create 1 rle with old method : 0.018647193908691406 length of segment : 171 time for calcul the mask position with numpy : 0.0013041496276855469 nb_pixel_total : 35377 time to create 1 rle with old method : 0.043763160705566406 length of segment : 462 time for calcul the mask position with numpy : 0.011754751205444336 nb_pixel_total : 483107 time to create 1 rle with new method : 0.03208422660827637 length of segment : 916 time for calcul the mask position with numpy : 0.0013988018035888672 nb_pixel_total : 23800 time to create 1 rle with old method : 0.03178119659423828 length of segment : 145 time for calcul the mask position with numpy : 0.00041222572326660156 nb_pixel_total : 7765 time to create 1 rle with old method : 0.009604454040527344 length of segment : 78 time for calcul the mask position with numpy : 0.0017011165618896484 nb_pixel_total : 33523 time to create 1 rle with old method : 0.037690162658691406 length of segment : 256 time for calcul the mask position with numpy : 0.001720428466796875 nb_pixel_total : 34556 time to create 1 rle with old method : 0.03959345817565918 length of segment : 202 time for calcul the mask position with numpy : 0.0014650821685791016 nb_pixel_total : 27870 time to create 1 rle with old method : 0.044326066970825195 length of segment : 236 time for calcul the mask position with numpy : 0.0009801387786865234 nb_pixel_total : 20516 time to create 1 rle with old method : 0.02636551856994629 length of segment : 138 time for calcul the mask position with numpy : 0.0015263557434082031 nb_pixel_total : 43871 time to create 1 rle with old method : 0.05504465103149414 length of segment : 266 time for calcul the mask position with numpy : 0.0014538764953613281 nb_pixel_total : 19993 time to create 1 rle with old method : 0.02628302574157715 length of segment : 234 time for calcul the mask position with numpy : 0.005710124969482422 nb_pixel_total : 141862 time to create 1 rle with old method : 0.18950295448303223 length of segment : 726 time for calcul the mask position with numpy : 0.004062652587890625 nb_pixel_total : 122243 time to create 1 rle with old method : 0.14937162399291992 length of segment : 244 time for calcul the mask position with numpy : 0.0034492015838623047 nb_pixel_total : 94144 time to create 1 rle with old method : 0.11342406272888184 length of segment : 498 time for calcul the mask position with numpy : 0.0044097900390625 nb_pixel_total : 88319 time to create 1 rle with old method : 0.10915946960449219 length of segment : 341 time for calcul the mask position with numpy : 0.003912687301635742 nb_pixel_total : 67433 time to create 1 rle with old method : 0.08207392692565918 length of segment : 496 time for calcul the mask position with numpy : 0.0005993843078613281 nb_pixel_total : 25095 time to create 1 rle with old method : 0.02881622314453125 length of segment : 161 time for calcul the mask position with numpy : 0.01949620246887207 nb_pixel_total : 290704 time to create 1 rle with new method : 0.06729531288146973 length of segment : 1052 time for calcul the mask position with numpy : 0.001737833023071289 nb_pixel_total : 58044 time to create 1 rle with old method : 0.07890486717224121 length of segment : 212 time for calcul the mask position with numpy : 0.0035293102264404297 nb_pixel_total : 144384 time to create 1 rle with old method : 0.16838383674621582 length of segment : 418 time for calcul the mask position with numpy : 0.001096963882446289 nb_pixel_total : 64541 time to create 1 rle with old method : 0.07863426208496094 length of segment : 360 time for calcul the mask position with numpy : 0.0019397735595703125 nb_pixel_total : 72399 time to create 1 rle with old method : 0.09453916549682617 length of segment : 177 time for calcul the mask position with numpy : 0.0064198970794677734 nb_pixel_total : 255442 time to create 1 rle with new method : 0.01779651641845703 length of segment : 749 time for calcul the mask position with numpy : 0.0004296302795410156 nb_pixel_total : 12396 time to create 1 rle with old method : 0.014587640762329102 length of segment : 187 time for calcul the mask position with numpy : 0.0018124580383300781 nb_pixel_total : 84591 time to create 1 rle with old method : 0.09992814064025879 length of segment : 266 time for calcul the mask position with numpy : 0.002474546432495117 nb_pixel_total : 165087 time to create 1 rle with new method : 0.005261659622192383 length of segment : 531 time for calcul the mask position with numpy : 0.00029277801513671875 nb_pixel_total : 13151 time to create 1 rle with old method : 0.015361785888671875 length of segment : 88 time for calcul the mask position with numpy : 0.005151033401489258 nb_pixel_total : 97021 time to create 1 rle with old method : 0.11573076248168945 length of segment : 408 time spent for convertir_results : 13.488081455230713 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 115 chid ids of type : 3594 Number RLEs to save : 36059 save missing photos in datou_result : time spend for datou_step_exec : 57.232423305511475 time spend to save output : 6.542668342590332 total time spend for step 1 : 63.77509164810181 step2:crop_condition Sat Apr 19 01:31:34 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 8 ! batch 1 Loaded 115 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 71 About to insert : list_path_to_insert length 71 new photo from crops ! About to upload 71 photos upload in portfolio : 3736932 init cache_photo without model_param we have 71 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1745019118_1944000 we have uploaded 71 photos in the portfolio 3736932 time of upload the photos Elapsed time : 24.09945845603943 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 17 About to insert : list_path_to_insert length 17 new photo from crops ! About to upload 17 photos upload in portfolio : 3736932 init cache_photo without model_param we have 17 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1745019152_1944000 we have uploaded 17 photos in the portfolio 3736932 time of upload the photos Elapsed time : 4.8544602394104 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3736932 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1745019158_1944000 we have uploaded 2 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.8417081832885742 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 18 About to insert : list_path_to_insert length 18 new photo from crops ! About to upload 18 photos upload in portfolio : 3736932 init cache_photo without model_param we have 18 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1745019168_1944000 we have uploaded 18 photos in the portfolio 3736932 time of upload the photos Elapsed time : 5.328834295272827 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 3736932 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1745019176_1944000 we have uploaded 4 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.3685498237609863 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3736932 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1745019179_1944000 we have uploaded 2 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.067430019378662 we have finished the crop for the class : pehd begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1745019181_1944000 we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.6941571235656738 we have finished the crop for the class : pet_fonce delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1351383053, 1351382952, 1351342354, 1351342290, 1351342202, 1351342120, 1351342036, 1351341951] Looping around the photos to save general results len do output : 115 /1351557471Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557475Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557478Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557482Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557486Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557491Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557494Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557498Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557503Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557506Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557510Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557513Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557516Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557518Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557522Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557525Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557527Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557531Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557533Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557536Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557540Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557542Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557545Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557547Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557550Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557553Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557556Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557558Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557561Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557563Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557565Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557568Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557570Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557574Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557576Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557579Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557581Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557584Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557587Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557590Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557592Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557596Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557598Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557601Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557604Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557606Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557609Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557611Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557614Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557616Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557618Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557621Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557624Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557627Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557629Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557631Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557634Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557636Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557639Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557642Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557644Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557650Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557652Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557656Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557658Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557661Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557664Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557666Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557671Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557673Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557676Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557925Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557928Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557929Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557932Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557934Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557936Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557939Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557940Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557943Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557945Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557948Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557950Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557953Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557954Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557957Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557959Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557961Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557995Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351557998Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351558464Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351558465Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351558468Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351558470Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351558471Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351558474Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351558476Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351558477Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351558480Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351558482Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351558483Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351558486Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351558488Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351558489Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351558492Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351558494Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351558495Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351558497Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351558588Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351558592Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351558596Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351558600Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351558689Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351558691Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1351558720Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351383053', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351382952', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351342354', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351342290', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351342202', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351342120', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351342036', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351341951', None, None, None, None, None, '2739209') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 353 time used for this insertion : 0.06563162803649902 save_final save missing photos in datou_result : time spend for datou_step_exec : 87.80626392364502 time spend to save output : 0.07002520561218262 total time spend for step 2 : 87.8762891292572 step3:rle_unique_nms_with_priority Sat Apr 19 01:33:02 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 115 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 10 nb_hashtags : 3 time to prepare the origin masks : 5.8540589809417725 time for calcul the mask position with numpy : 0.34342193603515625 nb_pixel_total : 5126059 time to create 1 rle with new method : 1.0162482261657715 time for calcul the mask position with numpy : 0.036415815353393555 nb_pixel_total : 76313 time to create 1 rle with old method : 0.09078001976013184 time for calcul the mask position with numpy : 0.03893303871154785 nb_pixel_total : 8522 time to create 1 rle with old method : 0.013762474060058594 time for calcul the mask position with numpy : 0.04192924499511719 nb_pixel_total : 30612 time to create 1 rle with old method : 0.036995649337768555 time for calcul the mask position with numpy : 0.04896259307861328 nb_pixel_total : 535412 time to create 1 rle with new method : 0.33712339401245117 time for calcul the mask position with numpy : 0.03706789016723633 nb_pixel_total : 39624 time to create 1 rle with old method : 0.04765605926513672 time for calcul the mask position with numpy : 0.038031578063964844 nb_pixel_total : 100733 time to create 1 rle with old method : 0.1215677261352539 time for calcul the mask position with numpy : 0.04343366622924805 nb_pixel_total : 382449 time to create 1 rle with new method : 1.1637427806854248 time for calcul the mask position with numpy : 0.03538060188293457 nb_pixel_total : 41787 time to create 1 rle with old method : 0.047533512115478516 time for calcul the mask position with numpy : 0.044071197509765625 nb_pixel_total : 692546 time to create 1 rle with new method : 0.6771738529205322 time for calcul the mask position with numpy : 0.03481578826904297 nb_pixel_total : 16183 time to create 1 rle with old method : 0.0186769962310791 create new chi : 4.41580057144165 time to delete rle : 0.015228033065795898 batch 1 Loaded 21 chid ids of type : 3594 ++++++++++++++++Number RLEs to save : 9210 TO DO : save crop sub photo not yet done ! save time : 1.4954800605773926 nb_obj : 6 nb_hashtags : 2 time to prepare the origin masks : 2.455764055252075 time for calcul the mask position with numpy : 0.40246009826660156 nb_pixel_total : 6102762 time to create 1 rle with new method : 0.509521484375 time for calcul the mask position with numpy : 0.022681474685668945 nb_pixel_total : 145337 time to create 1 rle with old method : 0.17629051208496094 time for calcul the mask position with numpy : 0.025163888931274414 nb_pixel_total : 332848 time to create 1 rle with new method : 0.4768359661102295 time for calcul the mask position with numpy : 0.02353835105895996 nb_pixel_total : 260935 time to create 1 rle with new method : 0.521956205368042 time for calcul the mask position with numpy : 0.03516364097595215 nb_pixel_total : 73436 time to create 1 rle with old method : 0.09113860130310059 time for calcul the mask position with numpy : 0.034790992736816406 nb_pixel_total : 101602 time to create 1 rle with old method : 0.12310290336608887 time for calcul the mask position with numpy : 0.034272193908691406 nb_pixel_total : 33320 time to create 1 rle with old method : 0.03692173957824707 create new chi : 2.5989010334014893 time to delete rle : 0.0022857189178466797 batch 1 Loaded 13 chid ids of type : 3594 ++++++++++++Number RLEs to save : 8792 TO DO : save crop sub photo not yet done ! save time : 1.5859251022338867 nb_obj : 20 nb_hashtags : 4 time to prepare the origin masks : 6.1087188720703125 time for calcul the mask position with numpy : 0.48369598388671875 nb_pixel_total : 6638018 time to create 1 rle with new method : 0.7204852104187012 time for calcul the mask position with numpy : 0.033933162689208984 nb_pixel_total : 9033 time to create 1 rle with old method : 0.009817838668823242 time for calcul the mask position with numpy : 0.035481929779052734 nb_pixel_total : 29834 time to create 1 rle with old method : 0.035897016525268555 time for calcul the mask position with numpy : 0.03425717353820801 nb_pixel_total : 47083 time to create 1 rle with old method : 0.055149078369140625 time for calcul the mask position with numpy : 0.03400826454162598 nb_pixel_total : 17641 time to create 1 rle with old method : 0.01979684829711914 time for calcul the mask position with numpy : 0.035845279693603516 nb_pixel_total : 8700 time to create 1 rle with old method : 0.010271310806274414 time for calcul the mask position with numpy : 0.033566951751708984 nb_pixel_total : 21837 time to create 1 rle with old method : 0.023755311965942383 time for calcul the mask position with numpy : 0.035770416259765625 nb_pixel_total : 13623 time to create 1 rle with old method : 0.015199422836303711 time for calcul the mask position with numpy : 0.033919334411621094 nb_pixel_total : 12876 time to create 1 rle with old method : 0.015877723693847656 time for calcul the mask position with numpy : 0.03243517875671387 nb_pixel_total : 43370 time to create 1 rle with old method : 0.048767805099487305 time for calcul the mask position with numpy : 0.03014969825744629 nb_pixel_total : 23236 time to create 1 rle with old method : 0.028099775314331055 time for calcul the mask position with numpy : 0.03306150436401367 nb_pixel_total : 7877 time to create 1 rle with old method : 0.009606599807739258 time for calcul the mask position with numpy : 0.034107208251953125 nb_pixel_total : 35995 time to create 1 rle with old method : 0.039745330810546875 time for calcul the mask position with numpy : 0.034479379653930664 nb_pixel_total : 19167 time to create 1 rle with old method : 0.023317337036132812 time for calcul the mask position with numpy : 0.03582191467285156 nb_pixel_total : 14585 time to create 1 rle with old method : 0.017316818237304688 time for calcul the mask position with numpy : 0.03757667541503906 nb_pixel_total : 13060 time to create 1 rle with old method : 0.014346122741699219 time for calcul the mask position with numpy : 0.03406381607055664 nb_pixel_total : 9676 time to create 1 rle with old method : 0.01052546501159668 time for calcul the mask position with numpy : 0.03455638885498047 nb_pixel_total : 7527 time to create 1 rle with old method : 0.009084701538085938 time for calcul the mask position with numpy : 0.03652000427246094 nb_pixel_total : 32192 time to create 1 rle with old method : 0.041706085205078125 time for calcul the mask position with numpy : 0.03391432762145996 nb_pixel_total : 36418 time to create 1 rle with old method : 0.03918313980102539 time for calcul the mask position with numpy : 0.024685382843017578 nb_pixel_total : 8492 time to create 1 rle with old method : 0.009317874908447266 create new chi : 2.3983068466186523 time to delete rle : 0.0015854835510253906 batch 1 Loaded 41 chid ids of type : 3594 ++++++++++++++++++++++Number RLEs to save : 9708 TO DO : save crop sub photo not yet done ! save time : 1.102860450744629 nb_obj : 17 nb_hashtags : 3 time to prepare the origin masks : 5.548927068710327 time for calcul the mask position with numpy : 0.27002906799316406 nb_pixel_total : 6724465 time to create 1 rle with new method : 0.47951436042785645 time for calcul the mask position with numpy : 0.04462265968322754 nb_pixel_total : 7774 time to create 1 rle with old method : 0.013560295104980469 time for calcul the mask position with numpy : 0.04226946830749512 nb_pixel_total : 40332 time to create 1 rle with old method : 0.050179481506347656 time for calcul the mask position with numpy : 0.03560757637023926 nb_pixel_total : 5452 time to create 1 rle with old method : 0.007760763168334961 time for calcul the mask position with numpy : 0.03614687919616699 nb_pixel_total : 3429 time to create 1 rle with old method : 0.0040283203125 time for calcul the mask position with numpy : 0.026790618896484375 nb_pixel_total : 14250 time to create 1 rle with old method : 0.027157068252563477 time for calcul the mask position with numpy : 0.02218341827392578 nb_pixel_total : 8503 time to create 1 rle with old method : 0.00967860221862793 time for calcul the mask position with numpy : 0.02415943145751953 nb_pixel_total : 13522 time to create 1 rle with old method : 0.020380735397338867 time for calcul the mask position with numpy : 0.024141550064086914 nb_pixel_total : 47512 time to create 1 rle with old method : 0.05832529067993164 time for calcul the mask position with numpy : 0.023810386657714844 nb_pixel_total : 9460 time to create 1 rle with old method : 0.011731863021850586 time for calcul the mask position with numpy : 0.022754430770874023 nb_pixel_total : 14965 time to create 1 rle with old method : 0.017731904983520508 time for calcul the mask position with numpy : 0.025257587432861328 nb_pixel_total : 27411 time to create 1 rle with old method : 0.04550433158874512 time for calcul the mask position with numpy : 0.02462291717529297 nb_pixel_total : 18338 time to create 1 rle with old method : 0.025256872177124023 time for calcul the mask position with numpy : 0.023712873458862305 nb_pixel_total : 41742 time to create 1 rle with old method : 0.052861928939819336 time for calcul the mask position with numpy : 0.030319929122924805 nb_pixel_total : 8157 time to create 1 rle with old method : 0.014964103698730469 time for calcul the mask position with numpy : 0.043311357498168945 nb_pixel_total : 18070 time to create 1 rle with old method : 0.020105838775634766 time for calcul the mask position with numpy : 0.03697037696838379 nb_pixel_total : 9951 time to create 1 rle with old method : 0.011500120162963867 time for calcul the mask position with numpy : 0.03706955909729004 nb_pixel_total : 36907 time to create 1 rle with old method : 0.044770002365112305 create new chi : 1.7520818710327148 time to delete rle : 0.0017955303192138672 batch 1 Loaded 35 chid ids of type : 3594 ++++++++++++++++++++++++Number RLEs to save : 9394 TO DO : save crop sub photo not yet done ! save time : 1.243659257888794 nb_obj : 25 nb_hashtags : 4 time to prepare the origin masks : 10.320309400558472 time for calcul the mask position with numpy : 0.7076878547668457 nb_pixel_total : 5801397 time to create 1 rle with new method : 0.7063694000244141 time for calcul the mask position with numpy : 0.022661685943603516 nb_pixel_total : 25052 time to create 1 rle with old method : 0.028098106384277344 time for calcul the mask position with numpy : 0.022571325302124023 nb_pixel_total : 17834 time to create 1 rle with old method : 0.01959848403930664 time for calcul the mask position with numpy : 0.023917675018310547 nb_pixel_total : 78421 time to create 1 rle with old method : 0.0942068099975586 time for calcul the mask position with numpy : 0.021946430206298828 nb_pixel_total : 2220 time to create 1 rle with old method : 0.002691507339477539 time for calcul the mask position with numpy : 0.021645784378051758 nb_pixel_total : 18972 time to create 1 rle with old method : 0.024640798568725586 time for calcul the mask position with numpy : 0.021799802780151367 nb_pixel_total : 58194 time to create 1 rle with old method : 0.06482911109924316 time for calcul the mask position with numpy : 0.02367234230041504 nb_pixel_total : 11130 time to create 1 rle with old method : 0.012549638748168945 time for calcul the mask position with numpy : 0.0309298038482666 nb_pixel_total : 81505 time to create 1 rle with old method : 0.09006953239440918 time for calcul the mask position with numpy : 0.030438899993896484 nb_pixel_total : 14690 time to create 1 rle with old method : 0.01648855209350586 time for calcul the mask position with numpy : 0.035817861557006836 nb_pixel_total : 44047 time to create 1 rle with old method : 0.04778265953063965 time for calcul the mask position with numpy : 0.03577113151550293 nb_pixel_total : 25282 time to create 1 rle with old method : 0.03159332275390625 time for calcul the mask position with numpy : 0.034088134765625 nb_pixel_total : 6236 time to create 1 rle with old method : 0.0067598819732666016 time for calcul the mask position with numpy : 0.03298163414001465 nb_pixel_total : 14631 time to create 1 rle with old method : 0.01619410514831543 time for calcul the mask position with numpy : 0.03330063819885254 nb_pixel_total : 24899 time to create 1 rle with old method : 0.030753612518310547 time for calcul the mask position with numpy : 0.03583240509033203 nb_pixel_total : 285358 time to create 1 rle with new method : 0.49529433250427246 time for calcul the mask position with numpy : 0.03500676155090332 nb_pixel_total : 28535 time to create 1 rle with old method : 0.031184911727905273 time for calcul the mask position with numpy : 0.035782814025878906 nb_pixel_total : 78666 time to create 1 rle with old method : 0.09270954132080078 time for calcul the mask position with numpy : 0.03408050537109375 nb_pixel_total : 48994 time to create 1 rle with old method : 0.05666851997375488 time for calcul the mask position with numpy : 0.033174991607666016 nb_pixel_total : 17244 time to create 1 rle with old method : 0.019392967224121094 time for calcul the mask position with numpy : 0.035540103912353516 nb_pixel_total : 158306 time to create 1 rle with new method : 0.4533677101135254 time for calcul the mask position with numpy : 0.03441643714904785 nb_pixel_total : 32842 time to create 1 rle with old method : 0.0403749942779541 time for calcul the mask position with numpy : 0.037481069564819336 nb_pixel_total : 56756 time to create 1 rle with old method : 0.06320500373840332 time for calcul the mask position with numpy : 0.035315513610839844 nb_pixel_total : 9746 time to create 1 rle with old method : 0.010696887969970703 time for calcul the mask position with numpy : 0.035307884216308594 nb_pixel_total : 92727 time to create 1 rle with old method : 0.10258293151855469 time for calcul the mask position with numpy : 0.03811979293823242 nb_pixel_total : 16556 time to create 1 rle with old method : 0.01795196533203125 create new chi : 4.1628642082214355 time to delete rle : 0.003715991973876953 batch 1 Loaded 51 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++Number RLEs to save : 16687 TO DO : save crop sub photo not yet done ! save time : 4.303417444229126 nb_obj : 16 nb_hashtags : 4 time to prepare the origin masks : 5.6747729778289795 time for calcul the mask position with numpy : 0.48675966262817383 nb_pixel_total : 5913057 time to create 1 rle with new method : 0.5303440093994141 time for calcul the mask position with numpy : 0.03989386558532715 nb_pixel_total : 34556 time to create 1 rle with old method : 0.04782724380493164 time for calcul the mask position with numpy : 0.04042959213256836 nb_pixel_total : 33523 time to create 1 rle with old method : 0.03867650032043457 time for calcul the mask position with numpy : 0.03394269943237305 nb_pixel_total : 7765 time to create 1 rle with old method : 0.009068965911865234 time for calcul the mask position with numpy : 0.04200243949890137 nb_pixel_total : 23800 time to create 1 rle with old method : 0.0385894775390625 time for calcul the mask position with numpy : 0.03759479522705078 nb_pixel_total : 5432 time to create 1 rle with old method : 0.0071027278900146484 time for calcul the mask position with numpy : 0.033574581146240234 nb_pixel_total : 35377 time to create 1 rle with old method : 0.04223942756652832 time for calcul the mask position with numpy : 0.022883176803588867 nb_pixel_total : 16197 time to create 1 rle with old method : 0.020030736923217773 time for calcul the mask position with numpy : 0.02505970001220703 nb_pixel_total : 15998 time to create 1 rle with old method : 0.026131391525268555 time for calcul the mask position with numpy : 0.029497385025024414 nb_pixel_total : 484003 time to create 1 rle with new method : 0.6194028854370117 time for calcul the mask position with numpy : 0.02592301368713379 nb_pixel_total : 53434 time to create 1 rle with old method : 0.0840916633605957 time for calcul the mask position with numpy : 0.021544694900512695 nb_pixel_total : 55590 time to create 1 rle with old method : 0.060187578201293945 time for calcul the mask position with numpy : 0.02720928192138672 nb_pixel_total : 128470 time to create 1 rle with old method : 0.14258098602294922 time for calcul the mask position with numpy : 0.026308059692382812 nb_pixel_total : 77648 time to create 1 rle with old method : 0.08828568458557129 time for calcul the mask position with numpy : 0.02580118179321289 nb_pixel_total : 84522 time to create 1 rle with old method : 0.09677386283874512 time for calcul the mask position with numpy : 0.041136741638183594 nb_pixel_total : 36400 time to create 1 rle with old method : 0.04056191444396973 time for calcul the mask position with numpy : 0.036841392517089844 nb_pixel_total : 44468 time to create 1 rle with old method : 0.04923510551452637 create new chi : 3.0037851333618164 time to delete rle : 0.0020170211791992188 batch 1 Loaded 33 chid ids of type : 3594 ++++++++++++++++++++++++++Number RLEs to save : 12641 TO DO : save crop sub photo not yet done ! save time : 2.718721866607666 nb_obj : 11 nb_hashtags : 4 time to prepare the origin masks : 4.47349214553833 time for calcul the mask position with numpy : 0.45594215393066406 nb_pixel_total : 6109889 time to create 1 rle with new method : 0.3096475601196289 time for calcul the mask position with numpy : 0.02247929573059082 nb_pixel_total : 290704 time to create 1 rle with new method : 0.5868899822235107 time for calcul the mask position with numpy : 0.02027583122253418 nb_pixel_total : 23660 time to create 1 rle with old method : 0.026862621307373047 time for calcul the mask position with numpy : 0.02111029624938965 nb_pixel_total : 67384 time to create 1 rle with old method : 0.08022189140319824 time for calcul the mask position with numpy : 0.022096872329711914 nb_pixel_total : 88104 time to create 1 rle with old method : 0.10351037979125977 time for calcul the mask position with numpy : 0.02166604995727539 nb_pixel_total : 94144 time to create 1 rle with old method : 0.10946512222290039 time for calcul the mask position with numpy : 0.023174762725830078 nb_pixel_total : 122243 time to create 1 rle with old method : 0.14028239250183105 time for calcul the mask position with numpy : 0.022257089614868164 nb_pixel_total : 141862 time to create 1 rle with old method : 0.15228986740112305 time for calcul the mask position with numpy : 0.021855592727661133 nb_pixel_total : 19993 time to create 1 rle with old method : 0.021909236907958984 time for calcul the mask position with numpy : 0.02199554443359375 nb_pixel_total : 43871 time to create 1 rle with old method : 0.05341219902038574 time for calcul the mask position with numpy : 0.03405618667602539 nb_pixel_total : 20516 time to create 1 rle with old method : 0.0235445499420166 time for calcul the mask position with numpy : 0.033849477767944336 nb_pixel_total : 27870 time to create 1 rle with old method : 0.03275799751281738 create new chi : 2.423349380493164 time to delete rle : 0.001926422119140625 batch 1 Loaded 23 chid ids of type : 3594 ++++++++++++++Number RLEs to save : 10790 TO DO : save crop sub photo not yet done ! save time : 0.9958670139312744 nb_obj : 10 nb_hashtags : 4 time to prepare the origin masks : 3.3307580947875977 time for calcul the mask position with numpy : 0.37441372871398926 nb_pixel_total : 6084708 time to create 1 rle with new method : 0.4682655334472656 time for calcul the mask position with numpy : 0.034697532653808594 nb_pixel_total : 97021 time to create 1 rle with old method : 0.11530613899230957 time for calcul the mask position with numpy : 0.034323692321777344 nb_pixel_total : 12839 time to create 1 rle with old method : 0.015612363815307617 time for calcul the mask position with numpy : 0.022304058074951172 nb_pixel_total : 165087 time to create 1 rle with new method : 0.3263978958129883 time for calcul the mask position with numpy : 0.023684978485107422 nb_pixel_total : 84591 time to create 1 rle with old method : 0.11891865730285645 time for calcul the mask position with numpy : 0.021632909774780273 nb_pixel_total : 12396 time to create 1 rle with old method : 0.01425933837890625 time for calcul the mask position with numpy : 0.022759437561035156 nb_pixel_total : 255442 time to create 1 rle with new method : 0.8429110050201416 time for calcul the mask position with numpy : 0.029994726181030273 nb_pixel_total : 72399 time to create 1 rle with old method : 0.08361577987670898 time for calcul the mask position with numpy : 0.021842241287231445 nb_pixel_total : 63329 time to create 1 rle with old method : 0.07645463943481445 time for calcul the mask position with numpy : 0.022992849349975586 nb_pixel_total : 144384 time to create 1 rle with old method : 0.19480490684509277 time for calcul the mask position with numpy : 0.023357629776000977 nb_pixel_total : 58044 time to create 1 rle with old method : 0.06871962547302246 create new chi : 3.045912265777588 time to delete rle : 0.001451253890991211 batch 1 Loaded 21 chid ids of type : 3594 +++++++++++++Number RLEs to save : 8793 TO DO : save crop sub photo not yet done ! save time : 1.1606123447418213 map_output_result : {1351383053: (0.0, 'Should be the crop_list due to order', 0), 1351382952: (0.0, 'Should be the crop_list due to order', 0), 1351342354: (0.0, 'Should be the crop_list due to order', 0), 1351342290: (0.0, 'Should be the crop_list due to order', 0), 1351342202: (0.0, 'Should be the crop_list due to order', 0), 1351342120: (0.0, 'Should be the crop_list due to order', 0), 1351342036: (0.0, 'Should be the crop_list due to order', 0), 1351341951: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1351383053, 1351382952, 1351342354, 1351342290, 1351342202, 1351342120, 1351342036, 1351341951] Looping around the photos to save general results len do output : 8 /1351383053.Didn't retrieve data . /1351382952.Didn't retrieve data . /1351342354.Didn't retrieve data . /1351342290.Didn't retrieve data . /1351342202.Didn't retrieve data . /1351342120.Didn't retrieve data . /1351342036.Didn't retrieve data . /1351341951.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351383053', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351382952', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351342354', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351342290', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351342202', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351342120', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351342036', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351341951', None, None, None, None, None, '2739209') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 24 time used for this insertion : 0.06279802322387695 save_final save missing photos in datou_result : time spend for datou_step_exec : 83.23637533187866 time spend to save output : 0.06329894065856934 total time spend for step 3 : 83.29967427253723 step4:ventilate_hashtags_in_portfolio Sat Apr 19 01:34:25 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 22230710 get user id for portfolio 22230710 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22230710 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('mal_croppe','environnement','autre','pet_fonce','pet_clair','papier','background','metal','pehd','flou','carton')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22230710 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('mal_croppe','environnement','autre','pet_fonce','pet_clair','papier','background','metal','pehd','flou','carton')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22230710 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('mal_croppe','environnement','autre','pet_fonce','pet_clair','papier','background','metal','pehd','flou','carton')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/22233817,22233818,22233819,22233820,22233821,22233822,22233823,22233824,22233825,22233826,22233827?tags=mal_croppe,environnement,autre,pet_fonce,pet_clair,papier,background,metal,pehd,flou,carton Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1351383053, 1351382952, 1351342354, 1351342290, 1351342202, 1351342120, 1351342036, 1351341951] Looping around the photos to save general results len do output : 1 /22230710. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351383053', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351382952', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351342354', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351342290', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351342202', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351342120', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351342036', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351341951', None, None, None, None, None, '2739209') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.017157316207885742 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.03511643409729 time spend to save output : 0.01749730110168457 total time spend for step 4 : 2.0526137351989746 step5:final Sat Apr 19 01:34:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1351383053: ('0.1400939010586874',), 1351382952: ('0.1400939010586874',), 1351342354: ('0.1400939010586874',), 1351342290: ('0.1400939010586874',), 1351342202: ('0.1400939010586874',), 1351342120: ('0.1400939010586874',), 1351342036: ('0.1400939010586874',), 1351341951: ('0.1400939010586874',)} new output for save of step final : {1351383053: ('0.1400939010586874',), 1351382952: ('0.1400939010586874',), 1351342354: ('0.1400939010586874',), 1351342290: ('0.1400939010586874',), 1351342202: ('0.1400939010586874',), 1351342120: ('0.1400939010586874',), 1351342036: ('0.1400939010586874',), 1351341951: ('0.1400939010586874',)} [1351383053, 1351382952, 1351342354, 1351342290, 1351342202, 1351342120, 1351342036, 1351341951] Looping around the photos to save general results len do output : 8 /1351383053.Didn't retrieve data . /1351382952.Didn't retrieve data . /1351342354.Didn't retrieve data . /1351342290.Didn't retrieve data . /1351342202.Didn't retrieve data . /1351342120.Didn't retrieve data . /1351342036.Didn't retrieve data . /1351341951.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351383053', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351382952', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351342354', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351342290', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351342202', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351342120', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351342036', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351341951', None, None, None, None, None, '2739209') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 24 time used for this insertion : 0.019165515899658203 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.16431522369384766 time spend to save output : 0.01959228515625 total time spend for step 5 : 0.18390750885009766 step6:blur_detection Sat Apr 19 01:34:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1745019029_1944000_1351383053_740272af7cb699ed31f429adb703e826.jpg resize: (2160, 3264) 1351383053 0.06535731976874075 treat image : temp/1745019029_1944000_1351382952_0b90c08bff29243f637e226837ebb800.jpg resize: (2160, 3264) 1351382952 0.32479161463475154 treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4.jpg resize: (2160, 3264) 1351342354 -2.3398752217929855 treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177.jpg resize: (2160, 3264) 1351342290 0.06871303043716946 treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f.jpg resize: (2160, 3264) 1351342202 -4.154733009541196 treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766.jpg resize: (2160, 3264) 1351342120 -3.9443433544306647 treat image : temp/1745019029_1944000_1351342036_e9f2fa9588ba32a5ce6f9fc90150148d.jpg resize: (2160, 3264) 1351342036 -0.8737206293890344 treat image : temp/1745019029_1944000_1351341951_48c5aa7709da9f39a7c199ad4f90970c.jpg resize: (2160, 3264) 1351341951 0.4760473909401716 treat image : temp/1745019029_1944000_1351383053_740272af7cb699ed31f429adb703e826_rle_crop_3756929077_0.png resize: (332, 184) 1351557471 -1.754802836097672 treat image : temp/1745019029_1944000_1351383053_740272af7cb699ed31f429adb703e826_rle_crop_3756929084_0.png resize: (856, 1645) 1351557475 0.6904349273135785 treat image : temp/1745019029_1944000_1351383053_740272af7cb699ed31f429adb703e826_rle_crop_3756929076_0.png resize: (458, 1726) 1351557478 -1.8494681588060333 treat image : temp/1745019029_1944000_1351383053_740272af7cb699ed31f429adb703e826_rle_crop_3756929080_0.png resize: (191, 339) 1351557482 -0.2170489015439215 treat image : temp/1745019029_1944000_1351383053_740272af7cb699ed31f429adb703e826_rle_crop_3756929079_0.png resize: (218, 584) 1351557486 -2.1071075976585822 treat image : temp/1745019029_1944000_1351382952_0b90c08bff29243f637e226837ebb800_rle_crop_3756929086_0.png resize: (306, 451) 1351557491 -0.6623553726275958 treat image : temp/1745019029_1944000_1351382952_0b90c08bff29243f637e226837ebb800_rle_crop_3756929085_0.png resize: (255, 182) 1351557494 -0.46144311236138785 treat image : temp/1745019029_1944000_1351382952_0b90c08bff29243f637e226837ebb800_rle_crop_3756929089_0.png resize: (826, 712) 1351557498 -0.0029058782910272732 treat image : temp/1745019029_1944000_1351382952_0b90c08bff29243f637e226837ebb800_rle_crop_3756929090_0.png resize: (780, 390) 1351557503 1.4292139533697172 treat image : temp/1745019029_1944000_1351382952_0b90c08bff29243f637e226837ebb800_rle_crop_3756929087_0.png resize: (353, 311) 1351557506 0.442354906555174 treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929098_0.png resize: (197, 152) 1351557510 -2.3984072192041648 treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929093_0.png resize: (283, 164) 1351557513 -3.0878241867401055 treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929101_0.png resize: (199, 216) 1351557516 -1.7221355416525188 treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929107_0.png resize: (204, 123) 1351557518 -3.0093734896236493 treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929097_0.png resize: (134, 135) 1351557522 -1.6116217186228277 treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929104_0.png resize: (191, 82) 1351557525 -0.04347636917494936 treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929105_0.png resize: (212, 131) 1351557527 -2.849967887512609 treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929102_0.png resize: (222, 293) 1351557531 -3.0515023358564206 treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929099_0.png resize: (280, 233) 1351557533 -2.0723772389681065 treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929110_0.png resize: (151, 113) 1351557536 -2.190798917015711 treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929094_0.png resize: (108, 128) 1351557540 -1.0761720049755306 treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929096_0.png resize: (176, 105) 1351557542 0.41223016487347686 treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929106_0.png resize: (145, 89) 1351557545 -0.8335694323738521 treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929108_0.png resize: (246, 259) 1351557547 -1.9377070276756134 treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929091_0.png resize: (110, 122) 1351557550 -1.0654001956692 treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929126_0.png resize: (508, 187) 1351557553 -2.7518522764168 treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929124_0.png resize: (53, 79) 1351557556 2.231150296207238 treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929113_0.png resize: (182, 151) 1351557558 -1.5341218931717207 treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929123_0.png resize: (235, 99) 1351557561 -0.5493239272820316 treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929118_0.png resize: (183, 160) 1351557563 -2.3617390002143757 treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929120_0.png resize: (393, 201) 1351557565 -1.835731506796605 treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929121_0.png resize: (174, 106) 1351557568 -1.7075984069278765 treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929117_0.png resize: (180, 232) 1351557570 -0.8499966256832255 treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929127_0.png resize: (138, 110) 1351557574 1.198763599886944 treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929114_0.png resize: (119, 80) 1351557576 1.5354213281519975 treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929125_0.png resize: (89, 76) 1351557579 0.755933778527912 treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929115_0.png resize: (274, 279) 1351557581 -1.4199248045979709 treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929116_0.png resize: (192, 122) 1351557584 -0.636262908413998 treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929112_0.png resize: (111, 138) 1351557587 -0.2922721994009266 treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929137_0.png resize: (264, 155) 1351557590 -1.9146292952285293 treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929138_0.png resize: (791, 572) 1351557592 -2.4044420422142285 treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929141_0.png resize: (109, 93) 1351557596 -2.0054416708962015 treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929134_0.png resize: (196, 120) 1351557598 -2.3808294420285008 treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929152_0.png resize: (195, 262) 1351557601 -2.4797057916462073 treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929149_0.png resize: (318, 384) 1351557604 -3.483824195795957 treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929139_0.png resize: (281, 131) 1351557606 -1.8184660512821869 treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929130_0.png resize: (94, 117) 1351557609 -0.5121866059658348 treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929144_0.png resize: (136, 133) 1351557611 -1.4341839713806024 treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929142_0.png resize: (199, 184) 1351557614 -2.224152963259795 treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929135_0.png resize: (426, 236) 1351557616 -1.332106918450458 treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929150_0.png resize: (270, 413) 1351557618 -2.665293541605333 treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929140_0.png resize: (188, 203) 1351557621 -2.084978061186561 treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929133_0.png resize: (533, 613) 1351557624 -2.3905294222345175 treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929158_0.png resize: (369, 179) 1351557627 0.003672472274388234 treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929155_0.png resize: (342, 357) 1351557629 -1.4301730036429556 treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929165_0.png resize: (139, 295) 1351557631 -2.539664217252593 treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929166_0.png resize: (74, 161) 1351557634 -2.2460692553021837 treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929153_0.png resize: (198, 280) 1351557636 -1.8312914250894943 treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929156_0.png resize: (370, 366) 1351557639 -0.9764566219863525 treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929157_0.png resize: (256, 617) 1351557642 -1.3471927440277331 treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929167_0.png resize: (256, 227) 1351557644 -0.33653305847802856 treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929161_0.png resize: (102, 214) 1351557650 -2.9881041372451858 treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929164_0.png resize: (916, 760) 1351557652 -0.5364039357859364 treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929154_0.png resize: (384, 175) 1351557656 -2.199770066232182 treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929162_0.png resize: (148, 162) 1351557658 -1.8269959054690144 treat image : temp/1745019029_1944000_1351342036_e9f2fa9588ba32a5ce6f9fc90150148d_rle_crop_3756929178_0.png resize: (160, 214) 1351557661 0.050951380154079254 treat image : temp/1745019029_1944000_1351342036_e9f2fa9588ba32a5ce6f9fc90150148d_rle_crop_3756929169_0.png resize: (236, 262) 1351557664 -0.888940725520876 treat image : temp/1745019029_1944000_1351342036_e9f2fa9588ba32a5ce6f9fc90150148d_rle_crop_3756929171_0.png resize: (263, 270) 1351557666 1.6677770687627838 treat image : temp/1745019029_1944000_1351342036_e9f2fa9588ba32a5ce6f9fc90150148d_rle_crop_3756929172_0.png resize: (186, 191) 1351557671 -1.8285988749936262 treat image : temp/1745019029_1944000_1351341951_48c5aa7709da9f39a7c199ad4f90970c_rle_crop_3756929183_0.png resize: (177, 666) 1351557673 -2.6243060760012265 treat image : temp/1745019029_1944000_1351341951_48c5aa7709da9f39a7c199ad4f90970c_rle_crop_3756929188_0.png resize: (86, 206) 1351557676 -1.6122896982093644 treat image : temp/1745019029_1944000_1351383053_740272af7cb699ed31f429adb703e826_rle_crop_3756929081_0.png resize: (779, 1043) 1351557925 -0.039739337789896766 treat image : temp/1745019029_1944000_1351383053_740272af7cb699ed31f429adb703e826_rle_crop_3756929078_0.png resize: (570, 981) 1351557928 -0.5855558059462158 treat image : temp/1745019029_1944000_1351383053_740272af7cb699ed31f429adb703e826_rle_crop_3756929083_0.png resize: (149, 70) 1351557929 1.295475456407965 treat image : temp/1745019029_1944000_1351383053_740272af7cb699ed31f429adb703e826_rle_crop_3756929075_0.png resize: (117, 169) 1351557932 -0.08745273170153002 treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929092_0.png resize: (214, 287) 1351557934 -1.7620132027171627 treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929095_0.png resize: (125, 114) 1351557936 -0.5964597948557689 treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929103_0.png resize: (205, 114) 1351557939 -1.223496081774843 treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929111_0.png resize: (152, 350) 1351557940 -0.5783316769200014 treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929119_0.png resize: (120, 112) 1351557943 -0.12096217073094535 treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929143_0.png resize: (230, 267) 1351557945 -1.5293738562785135 treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929129_0.png resize: (510, 292) 1351557948 -1.6888844885691943 treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929151_0.png resize: (169, 177) 1351557950 -1.9738712224312835 treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929146_0.png resize: (113, 146) 1351557953 -1.3235600580635893 treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929160_0.png resize: (933, 758) 1351557954 -0.5049827879926749 treat image : temp/1745019029_1944000_1351342036_e9f2fa9588ba32a5ce6f9fc90150148d_rle_crop_3756929173_0.png resize: (726, 320) 1351557957 -0.03821726369147619 treat image : temp/1745019029_1944000_1351341951_48c5aa7709da9f39a7c199ad4f90970c_rle_crop_3756929186_0.png resize: (209, 574) 1351557959 -1.7744416430370766 treat image : temp/1745019029_1944000_1351341951_48c5aa7709da9f39a7c199ad4f90970c_rle_crop_3756929185_0.png resize: (101, 185) 1351557961 -2.6875917834450327 treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929100_0.png resize: (96, 95) 1351557995 -2.8235348986873325 treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929122_0.png resize: (110, 92) 1351557998 -1.529787874734325 treat image : temp/1745019029_1944000_1351383053_740272af7cb699ed31f429adb703e826_rle_crop_3756929082_0.png resize: (168, 272) 1351558464 -0.7818359970130082 treat image : temp/1745019029_1944000_1351382952_0b90c08bff29243f637e226837ebb800_rle_crop_3756929088_0.png resize: (633, 716) 1351558465 -1.6098833170884803 treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929128_0.png resize: (146, 176) 1351558468 -2.292285773860478 treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929147_0.png resize: (285, 261) 1351558470 -2.067000944527775 treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929145_0.png resize: (335, 378) 1351558471 -3.5307500727962475 treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929136_0.png resize: (309, 334) 1351558474 -2.4200962490791005 treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929148_0.png resize: (194, 172) 1351558476 -1.741762755581997 treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929168_0.png resize: (190, 248) 1351558477 -1.7370031381969173 treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929159_0.png resize: (304, 295) 1351558480 -3.4794349178586472 treat image : temp/1745019029_1944000_1351342036_e9f2fa9588ba32a5ce6f9fc90150148d_rle_crop_3756929174_0.png resize: (235, 726) 1351558482 -0.5176333464230878 treat image : temp/1745019029_1944000_1351342036_e9f2fa9588ba32a5ce6f9fc90150148d_rle_crop_3756929179_0.png resize: (998, 1062) 1351558483 -1.0679826158868644 treat image : temp/1745019029_1944000_1351342036_e9f2fa9588ba32a5ce6f9fc90150148d_rle_crop_3756929170_0.png resize: (130, 222) 1351558486 -0.32572254353584834 treat image : temp/1745019029_1944000_1351342036_e9f2fa9588ba32a5ce6f9fc90150148d_rle_crop_3756929176_0.png resize: (293, 483) 1351558488 -2.184623179915419 treat image : temp/1745019029_1944000_1351342036_e9f2fa9588ba32a5ce6f9fc90150148d_rle_crop_3756929175_0.png resize: (494, 287) 1351558489 -0.8538776927865441 treat image : temp/1745019029_1944000_1351341951_48c5aa7709da9f39a7c199ad4f90970c_rle_crop_3756929181_0.png resize: (415, 506) 1351558492 -0.9035586496949599 treat image : temp/1745019029_1944000_1351341951_48c5aa7709da9f39a7c199ad4f90970c_rle_crop_3756929180_0.png resize: (202, 469) 1351558494 -1.0593096405338238 treat image : temp/1745019029_1944000_1351341951_48c5aa7709da9f39a7c199ad4f90970c_rle_crop_3756929182_0.png resize: (303, 256) 1351558495 -1.0703204234286305 treat image : temp/1745019029_1944000_1351341951_48c5aa7709da9f39a7c199ad4f90970c_rle_crop_3756929184_0.png resize: (660, 707) 1351558497 -0.6885581126301643 treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929131_0.png resize: (285, 249) 1351558588 -1.406248182051281 treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929132_0.png resize: (214, 192) 1351558592 -1.6283008871153746 treat image : temp/1745019029_1944000_1351341951_48c5aa7709da9f39a7c199ad4f90970c_rle_crop_3756929189_0.png resize: (377, 424) 1351558596 -2.9525581819012965 treat image : temp/1745019029_1944000_1351341951_48c5aa7709da9f39a7c199ad4f90970c_rle_crop_3756929187_0.png resize: (499, 382) 1351558600 -0.30483383830045546 treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929163_0.png resize: (229, 249) 1351558689 -3.5731590043169694 treat image : temp/1745019029_1944000_1351342036_e9f2fa9588ba32a5ce6f9fc90150148d_rle_crop_3756929177_0.png resize: (495, 347) 1351558691 -0.7445792275172062 treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929109_0.png resize: (174, 255) 1351558720 -2.249408186198615 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 123 time used for this insertion : 0.01639389991760254 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 123 time used for this insertion : 0.7571582794189453 save missing photos in datou_result : time spend for datou_step_exec : 34.41395378112793 time spend to save output : 0.7789738178253174 total time spend for step 6 : 35.19292759895325 step7:brightness Sat Apr 19 01:35:02 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1745019029_1944000_1351383053_740272af7cb699ed31f429adb703e826.jpg treat image : temp/1745019029_1944000_1351382952_0b90c08bff29243f637e226837ebb800.jpg treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4.jpg treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177.jpg treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f.jpg treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766.jpg treat image : temp/1745019029_1944000_1351342036_e9f2fa9588ba32a5ce6f9fc90150148d.jpg treat image : temp/1745019029_1944000_1351341951_48c5aa7709da9f39a7c199ad4f90970c.jpg treat image : temp/1745019029_1944000_1351383053_740272af7cb699ed31f429adb703e826_rle_crop_3756929077_0.png treat image : temp/1745019029_1944000_1351383053_740272af7cb699ed31f429adb703e826_rle_crop_3756929084_0.png treat image : temp/1745019029_1944000_1351383053_740272af7cb699ed31f429adb703e826_rle_crop_3756929076_0.png treat image : temp/1745019029_1944000_1351383053_740272af7cb699ed31f429adb703e826_rle_crop_3756929080_0.png treat image : temp/1745019029_1944000_1351383053_740272af7cb699ed31f429adb703e826_rle_crop_3756929079_0.png treat image : temp/1745019029_1944000_1351382952_0b90c08bff29243f637e226837ebb800_rle_crop_3756929086_0.png treat image : temp/1745019029_1944000_1351382952_0b90c08bff29243f637e226837ebb800_rle_crop_3756929085_0.png treat image : temp/1745019029_1944000_1351382952_0b90c08bff29243f637e226837ebb800_rle_crop_3756929089_0.png treat image : temp/1745019029_1944000_1351382952_0b90c08bff29243f637e226837ebb800_rle_crop_3756929090_0.png treat image : temp/1745019029_1944000_1351382952_0b90c08bff29243f637e226837ebb800_rle_crop_3756929087_0.png treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929098_0.png treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929093_0.png treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929101_0.png treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929107_0.png treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929097_0.png treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929104_0.png treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929105_0.png treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929102_0.png treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929099_0.png treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929110_0.png treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929094_0.png treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929096_0.png treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929106_0.png treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929108_0.png treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929091_0.png treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929126_0.png treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929124_0.png treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929113_0.png treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929123_0.png treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929118_0.png treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929120_0.png treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929121_0.png treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929117_0.png treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929127_0.png treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929114_0.png treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929125_0.png treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929115_0.png treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929116_0.png treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929112_0.png treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929137_0.png treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929138_0.png treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929141_0.png treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929134_0.png treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929152_0.png treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929149_0.png treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929139_0.png treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929130_0.png treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929144_0.png treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929142_0.png treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929135_0.png treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929150_0.png treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929140_0.png treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929133_0.png treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929158_0.png treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929155_0.png treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929165_0.png treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929166_0.png treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929153_0.png treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929156_0.png treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929157_0.png treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929167_0.png treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929161_0.png treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929164_0.png treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929154_0.png treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929162_0.png treat image : temp/1745019029_1944000_1351342036_e9f2fa9588ba32a5ce6f9fc90150148d_rle_crop_3756929178_0.png treat image : temp/1745019029_1944000_1351342036_e9f2fa9588ba32a5ce6f9fc90150148d_rle_crop_3756929169_0.png treat image : temp/1745019029_1944000_1351342036_e9f2fa9588ba32a5ce6f9fc90150148d_rle_crop_3756929171_0.png treat image : temp/1745019029_1944000_1351342036_e9f2fa9588ba32a5ce6f9fc90150148d_rle_crop_3756929172_0.png treat image : temp/1745019029_1944000_1351341951_48c5aa7709da9f39a7c199ad4f90970c_rle_crop_3756929183_0.png treat image : temp/1745019029_1944000_1351341951_48c5aa7709da9f39a7c199ad4f90970c_rle_crop_3756929188_0.png treat image : temp/1745019029_1944000_1351383053_740272af7cb699ed31f429adb703e826_rle_crop_3756929081_0.png treat image : temp/1745019029_1944000_1351383053_740272af7cb699ed31f429adb703e826_rle_crop_3756929078_0.png treat image : temp/1745019029_1944000_1351383053_740272af7cb699ed31f429adb703e826_rle_crop_3756929083_0.png treat image : temp/1745019029_1944000_1351383053_740272af7cb699ed31f429adb703e826_rle_crop_3756929075_0.png treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929092_0.png treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929095_0.png treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929103_0.png treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929111_0.png treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929119_0.png treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929143_0.png treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929129_0.png treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929151_0.png treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929146_0.png treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929160_0.png treat image : temp/1745019029_1944000_1351342036_e9f2fa9588ba32a5ce6f9fc90150148d_rle_crop_3756929173_0.png treat image : temp/1745019029_1944000_1351341951_48c5aa7709da9f39a7c199ad4f90970c_rle_crop_3756929186_0.png treat image : temp/1745019029_1944000_1351341951_48c5aa7709da9f39a7c199ad4f90970c_rle_crop_3756929185_0.png treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929100_0.png treat image : temp/1745019029_1944000_1351342290_1f181718b6aebf455de8cd696bd17177_rle_crop_3756929122_0.png treat image : temp/1745019029_1944000_1351383053_740272af7cb699ed31f429adb703e826_rle_crop_3756929082_0.png treat image : temp/1745019029_1944000_1351382952_0b90c08bff29243f637e226837ebb800_rle_crop_3756929088_0.png treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929128_0.png treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929147_0.png treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929145_0.png treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929136_0.png treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929148_0.png treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929168_0.png treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929159_0.png treat image : temp/1745019029_1944000_1351342036_e9f2fa9588ba32a5ce6f9fc90150148d_rle_crop_3756929174_0.png treat image : temp/1745019029_1944000_1351342036_e9f2fa9588ba32a5ce6f9fc90150148d_rle_crop_3756929179_0.png treat image : temp/1745019029_1944000_1351342036_e9f2fa9588ba32a5ce6f9fc90150148d_rle_crop_3756929170_0.png treat image : temp/1745019029_1944000_1351342036_e9f2fa9588ba32a5ce6f9fc90150148d_rle_crop_3756929176_0.png treat image : temp/1745019029_1944000_1351342036_e9f2fa9588ba32a5ce6f9fc90150148d_rle_crop_3756929175_0.png treat image : temp/1745019029_1944000_1351341951_48c5aa7709da9f39a7c199ad4f90970c_rle_crop_3756929181_0.png treat image : temp/1745019029_1944000_1351341951_48c5aa7709da9f39a7c199ad4f90970c_rle_crop_3756929180_0.png treat image : temp/1745019029_1944000_1351341951_48c5aa7709da9f39a7c199ad4f90970c_rle_crop_3756929182_0.png treat image : temp/1745019029_1944000_1351341951_48c5aa7709da9f39a7c199ad4f90970c_rle_crop_3756929184_0.png treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929131_0.png treat image : temp/1745019029_1944000_1351342202_00a5c9e3c4f6f1d4452856fc8f62ec3f_rle_crop_3756929132_0.png treat image : temp/1745019029_1944000_1351341951_48c5aa7709da9f39a7c199ad4f90970c_rle_crop_3756929189_0.png treat image : temp/1745019029_1944000_1351341951_48c5aa7709da9f39a7c199ad4f90970c_rle_crop_3756929187_0.png treat image : temp/1745019029_1944000_1351342120_8583be0b48d962f54f137ae866a34766_rle_crop_3756929163_0.png treat image : temp/1745019029_1944000_1351342036_e9f2fa9588ba32a5ce6f9fc90150148d_rle_crop_3756929177_0.png treat image : temp/1745019029_1944000_1351342354_c189425d8c35e4d7953ae02ed9a339b4_rle_crop_3756929109_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 123 time used for this insertion : 0.018051862716674805 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 123 time used for this insertion : 0.04184079170227051 save missing photos in datou_result : time spend for datou_step_exec : 9.081742525100708 time spend to save output : 0.06551289558410645 total time spend for step 7 : 9.147255420684814 step8:velours_tree Sat Apr 19 01:35:11 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.24964332580566406 time spend to save output : 4.482269287109375e-05 total time spend for step 8 : 0.24968814849853516 step9:send_mail_cod Sat Apr 19 01:35:12 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P22230710_19-04-2025_01_35_12.pdf 22233817 imagette222338171745019312 22233819 change filename to text .change filename to text .change filename to text .change filename to text .imagette222338191745019312 22233820 change filename to text .imagette222338201745019312 22233821 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette222338211745019312 22233822 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette222338221745019313 22233823 imagette222338231745019315 22233824 change filename to text .change filename to text .imagette222338241745019315 22233825 change filename to text .change filename to text .imagette222338251745019315 22233826 imagette222338261745019315 22233827 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette222338271745019315 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=22230710 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/22233817,22233818,22233819,22233820,22233821,22233822,22233823,22233824,22233825,22233826,22233827?tags=mal_croppe,environnement,autre,pet_fonce,pet_clair,papier,background,metal,pehd,flou,carton args[1351383053] : ((1351383053, 0.06535731976874075, 492688767), (1351383053, 0.17717493881061647, 2107752395), '0.1400939010586874') We are sending mail with results at report@fotonower.com args[1351382952] : ((1351382952, 0.32479161463475154, 492688767), (1351382952, -0.3152528837157902, 496442774), '0.1400939010586874') We are sending mail with results at report@fotonower.com args[1351342354] : ((1351342354, -2.3398752217929855, 492609224), (1351342354, 0.7199070493077685, 2107752395), '0.1400939010586874') We are sending mail with results at report@fotonower.com args[1351342290] : ((1351342290, 0.06871303043716946, 492688767), (1351342290, 0.8863515363097756, 2107752395), '0.1400939010586874') We are sending mail with results at report@fotonower.com args[1351342202] : ((1351342202, -4.154733009541196, 492609224), (1351342202, 0.2326935988130616, 2107752395), '0.1400939010586874') We are sending mail with results at report@fotonower.com args[1351342120] : ((1351342120, -3.9443433544306647, 492609224), (1351342120, 0.21426113332978192, 2107752395), '0.1400939010586874') We are sending mail with results at report@fotonower.com args[1351342036] : ((1351342036, -0.8737206293890344, 492688767), (1351342036, 0.04970644927915438, 2107752395), '0.1400939010586874') We are sending mail with results at report@fotonower.com args[1351341951] : ((1351341951, 0.4760473909401716, 492688767), (1351341951, -0.13920597372198534, 496442774), '0.1400939010586874') We are sending mail with results at report@fotonower.com refus_total : 0.1400939010586874 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=22230710 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22230710_19-04-2025_01_35_12.pdf results_Auto_P22230710_19-04-2025_01_35_12.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22230710_19-04-2025_01_35_12.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','22230710','results_Auto_P22230710_19-04-2025_01_35_12.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22230710_19-04-2025_01_35_12.pdf','pdf','','0.88','0.1400939010586874') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/22230710

https://www.fotonower.com/image?json=false&list_photos_id=1351383053
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1351382952
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1351342354
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1351342290
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1351342202
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1351342120
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1351342036
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1351341951
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 14.01%
Veuillez trouver les photos des contaminants.

exemples de contaminants: autre: https://www.fotonower.com/view/22233819?limit=200
exemples de contaminants: pet_fonce: https://www.fotonower.com/view/22233820?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/22233821?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/22233822?limit=200
exemples de contaminants: metal: https://www.fotonower.com/view/22233824?limit=200
exemples de contaminants: pehd: https://www.fotonower.com/view/22233825?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/22233827?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22230710_19-04-2025_01_35_12.pdf.

Lien vers velours :https://www.fotonower.com/velours/22233817,22233818,22233819,22233820,22233821,22233822,22233823,22233824,22233825,22233826,22233827?tags=mal_croppe,environnement,autre,pet_fonce,pet_clair,papier,background,metal,pehd,flou,carton.


L'équipe Fotonower 202 b'' Server: nginx Date: Fri, 18 Apr 2025 23:35:20 GMT Content-Length: 0 Connection: close X-Message-Id: uLHhqXejTne1LE-ZrRlEYg Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1351383053, 1351382952, 1351342354, 1351342290, 1351342202, 1351342120, 1351342036, 1351341951] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351383053', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351382952', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351342354', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351342290', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351342202', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351342120', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351342036', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351341951', None, None, None, None, None, '2739209') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 8 time used for this insertion : 0.038231849670410156 save_final save missing photos in datou_result : time spend for datou_step_exec : 7.839770555496216 time spend to save output : 0.038419485092163086 total time spend for step 9 : 7.878190040588379 step10:split_time_score Sat Apr 19 01:35:20 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('20', 8),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 18042025 22230710 Nombre de photos uploadées : 8 / 23040 (0%) 18042025 22230710 Nombre de photos taguées (types de déchets): 0 / 8 (0%) 18042025 22230710 Nombre de photos taguées (volume) : 0 / 8 (0%) elapsed_time : load_data_split_time_score 3.0994415283203125e-06 elapsed_time : order_list_meta_photo_and_scores 7.62939453125e-06 ???????? elapsed_time : fill_and_build_computed_from_old_data 0.0005078315734863281 elapsed_time : insert_dashboard_record_day_entry 0.05601692199707031 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.05743480795288126 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22197665_19-04-2025_01_11_43.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22197665 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22197665 AND mptpi.`type`=3726 To do Qualite : 0.13163184947388776 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22230680_19-04-2025_01_19_14.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22230680 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22230680 AND mptpi.`type`=3594 To do Qualite : 0.03705996151466524 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22230683_19-04-2025_00_35_15.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22230683 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22230683 AND mptpi.`type`=3726 To do Qualite : 0.1671486077069717 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22230688_19-04-2025_00_58_06.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22230688 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22230688 AND mptpi.`type`=3594 To do Qualite : 0.16022285411560466 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22230691_19-04-2025_01_17_24.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22230691 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22230691 AND mptpi.`type`=3594 To do Qualite : 0.1295005226355248 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22230693_19-04-2025_00_42_22.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22230693 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22230693 AND mptpi.`type`=3594 To do Qualite : 0.049673804203741845 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22230696_19-04-2025_00_34_58.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22230696 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22230696 AND mptpi.`type`=3726 To do Qualite : 0.1331646621372048 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22222764_18-04-2025_21_27_57.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22222764 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22222764 AND mptpi.`type`=3594 To do Qualite : 0.1400939010586874 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22230710_19-04-2025_01_35_12.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22230710 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22230710 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'18042025': {'nb_upload': 8, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1351383053, 1351382952, 1351342354, 1351342290, 1351342202, 1351342120, 1351342036, 1351341951] Looping around the photos to save general results len do output : 1 /22230710Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351383053', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351382952', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351342354', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351342290', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351342202', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351342120', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351342036', None, None, None, None, None, '2739209') ('3318', None, None, None, None, None, None, None, '2739209') ('3318', '22230710', '1351341951', None, None, None, None, None, '2739209') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.015172004699707031 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.9112780094146729 time spend to save output : 0.015399932861328125 total time spend for step 10 : 0.926677942276001 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 8 set_done_treatment 133.38user 99.72system 4:55.54elapsed 78%CPU (0avgtext+0avgdata 4891020maxresident)k 157272inputs+93480outputs (8645major+11795693minor)pagefaults 0swaps