python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3823264 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['2564396'] with mtr_portfolio_ids : ['20334534'] and first list_photo_ids : [] new path : /proc/3823264/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 5 ; length of list_pids : 5 ; length of list_args : 5 time to download the photos : 1.0552644729614258 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Fri Feb 7 13:50:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10998 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-07 13:50:32.784425: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-07 13:50:32.811255: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-07 13:50:32.813071: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f5e38000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-07 13:50:32.813127: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-07 13:50:32.816290: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-07 13:50:33.106349: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xb7047a0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-07 13:50:33.106408: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-07 13:50:33.107850: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-07 13:50:33.108455: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-07 13:50:33.111752: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-07 13:50:33.114785: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-07 13:50:33.115356: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-07 13:50:33.118032: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-07 13:50:33.119192: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-07 13:50:33.123848: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-07 13:50:33.125390: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-07 13:50:33.125465: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-07 13:50:33.126276: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-07 13:50:33.126293: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-07 13:50:33.126302: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-07 13:50:33.127726: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10193 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-07 13:50:33.392047: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-07 13:50:33.392174: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-07 13:50:33.392202: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-07 13:50:33.392227: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-07 13:50:33.392253: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-07 13:50:33.392277: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-07 13:50:33.392301: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-07 13:50:33.392326: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-07 13:50:33.394019: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-07 13:50:33.395232: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-07 13:50:33.395271: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-07 13:50:33.395287: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-07 13:50:33.395301: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-07 13:50:33.395315: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-07 13:50:33.395329: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-07 13:50:33.395343: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-07 13:50:33.395357: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-07 13:50:33.396657: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-07 13:50:33.396686: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-07 13:50:33.396695: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-07 13:50:33.396703: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-07 13:50:33.398457: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10193 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-07 13:50:43.270276: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-07 13:50:43.480823: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 5 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 145.85000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 44 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.60000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 50 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 144.35000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 38 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 149.60000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 33 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 58 Detection mask done ! Trying to reset tf kernel 3823820 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5706 tf kernel not reseted sub process len(results) : 5 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 5 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10553 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0015053749084472656 nb_pixel_total : 41635 time to create 1 rle with old method : 0.0449681282043457 length of segment : 208 time for calcul the mask position with numpy : 0.0003845691680908203 nb_pixel_total : 8998 time to create 1 rle with old method : 0.010408878326416016 length of segment : 124 time for calcul the mask position with numpy : 0.0008974075317382812 nb_pixel_total : 21432 time to create 1 rle with old method : 0.033377885818481445 length of segment : 168 time for calcul the mask position with numpy : 0.0009605884552001953 nb_pixel_total : 25750 time to create 1 rle with old method : 0.028514623641967773 length of segment : 220 time for calcul the mask position with numpy : 0.0006020069122314453 nb_pixel_total : 19167 time to create 1 rle with old method : 0.021834373474121094 length of segment : 199 time for calcul the mask position with numpy : 0.001974344253540039 nb_pixel_total : 43278 time to create 1 rle with old method : 0.04999661445617676 length of segment : 209 time for calcul the mask position with numpy : 0.00048661231994628906 nb_pixel_total : 11750 time to create 1 rle with old method : 0.013783931732177734 length of segment : 160 time for calcul the mask position with numpy : 0.0008404254913330078 nb_pixel_total : 20937 time to create 1 rle with old method : 0.0246431827545166 length of segment : 162 time for calcul the mask position with numpy : 0.009124517440795898 nb_pixel_total : 182325 time to create 1 rle with new method : 0.014313697814941406 length of segment : 552 time for calcul the mask position with numpy : 0.0002243518829345703 nb_pixel_total : 6925 time to create 1 rle with old method : 0.012553930282592773 length of segment : 91 time for calcul the mask position with numpy : 0.0011358261108398438 nb_pixel_total : 30986 time to create 1 rle with old method : 0.0330808162689209 length of segment : 228 time for calcul the mask position with numpy : 0.000530242919921875 nb_pixel_total : 12739 time to create 1 rle with old method : 0.014806032180786133 length of segment : 197 time for calcul the mask position with numpy : 0.0010099411010742188 nb_pixel_total : 37266 time to create 1 rle with old method : 0.04082798957824707 length of segment : 273 time for calcul the mask position with numpy : 0.00022792816162109375 nb_pixel_total : 6498 time to create 1 rle with old method : 0.008348703384399414 length of segment : 65 time for calcul the mask position with numpy : 0.0003147125244140625 nb_pixel_total : 13382 time to create 1 rle with old method : 0.014891624450683594 length of segment : 134 time for calcul the mask position with numpy : 0.0036394596099853516 nb_pixel_total : 123797 time to create 1 rle with old method : 0.13656973838806152 length of segment : 418 time for calcul the mask position with numpy : 0.00026416778564453125 nb_pixel_total : 6848 time to create 1 rle with old method : 0.0075795650482177734 length of segment : 96 time for calcul the mask position with numpy : 0.006006002426147461 nb_pixel_total : 170821 time to create 1 rle with new method : 0.009842634201049805 length of segment : 557 time for calcul the mask position with numpy : 0.0011448860168457031 nb_pixel_total : 24773 time to create 1 rle with old method : 0.026489734649658203 length of segment : 205 time for calcul the mask position with numpy : 0.0005862712860107422 nb_pixel_total : 5629 time to create 1 rle with old method : 0.00694727897644043 length of segment : 87 time for calcul the mask position with numpy : 0.0013115406036376953 nb_pixel_total : 24799 time to create 1 rle with old method : 0.027790546417236328 length of segment : 231 time for calcul the mask position with numpy : 0.003253459930419922 nb_pixel_total : 56914 time to create 1 rle with old method : 0.06144070625305176 length of segment : 324 time for calcul the mask position with numpy : 0.00696563720703125 nb_pixel_total : 129857 time to create 1 rle with old method : 0.14623498916625977 length of segment : 388 time for calcul the mask position with numpy : 0.0006654262542724609 nb_pixel_total : 15723 time to create 1 rle with old method : 0.01760721206665039 length of segment : 230 time for calcul the mask position with numpy : 0.00548243522644043 nb_pixel_total : 145487 time to create 1 rle with old method : 0.15593695640563965 length of segment : 399 time for calcul the mask position with numpy : 0.050600528717041016 nb_pixel_total : 445383 time to create 1 rle with new method : 0.03561258316040039 length of segment : 897 time for calcul the mask position with numpy : 0.0005393028259277344 nb_pixel_total : 13848 time to create 1 rle with old method : 0.01610875129699707 length of segment : 279 time for calcul the mask position with numpy : 0.0018858909606933594 nb_pixel_total : 47934 time to create 1 rle with old method : 0.06421160697937012 length of segment : 259 time for calcul the mask position with numpy : 0.0004971027374267578 nb_pixel_total : 8799 time to create 1 rle with old method : 0.011190652847290039 length of segment : 102 time for calcul the mask position with numpy : 0.0009281635284423828 nb_pixel_total : 15847 time to create 1 rle with old method : 0.017442941665649414 length of segment : 280 time for calcul the mask position with numpy : 0.0015730857849121094 nb_pixel_total : 28204 time to create 1 rle with old method : 0.035341739654541016 length of segment : 179 time for calcul the mask position with numpy : 0.0021615028381347656 nb_pixel_total : 35920 time to create 1 rle with old method : 0.040009260177612305 length of segment : 326 time for calcul the mask position with numpy : 0.0008113384246826172 nb_pixel_total : 24417 time to create 1 rle with old method : 0.027102947235107422 length of segment : 127 time for calcul the mask position with numpy : 0.003774881362915039 nb_pixel_total : 94805 time to create 1 rle with old method : 0.1036384105682373 length of segment : 338 time for calcul the mask position with numpy : 0.0009126663208007812 nb_pixel_total : 19912 time to create 1 rle with old method : 0.02311253547668457 length of segment : 152 time for calcul the mask position with numpy : 0.0008292198181152344 nb_pixel_total : 13843 time to create 1 rle with old method : 0.015770912170410156 length of segment : 176 time for calcul the mask position with numpy : 0.0012874603271484375 nb_pixel_total : 30014 time to create 1 rle with old method : 0.03358912467956543 length of segment : 227 time for calcul the mask position with numpy : 0.003511190414428711 nb_pixel_total : 55273 time to create 1 rle with old method : 0.062297821044921875 length of segment : 408 time for calcul the mask position with numpy : 0.0017888545989990234 nb_pixel_total : 44203 time to create 1 rle with old method : 0.04889726638793945 length of segment : 291 time for calcul the mask position with numpy : 0.0007648468017578125 nb_pixel_total : 21921 time to create 1 rle with old method : 0.02663421630859375 length of segment : 109 time for calcul the mask position with numpy : 0.0007390975952148438 nb_pixel_total : 18625 time to create 1 rle with old method : 0.020853281021118164 length of segment : 158 time for calcul the mask position with numpy : 0.0005140304565429688 nb_pixel_total : 12515 time to create 1 rle with old method : 0.014219045639038086 length of segment : 126 time for calcul the mask position with numpy : 0.0022497177124023438 nb_pixel_total : 65752 time to create 1 rle with old method : 0.07231974601745605 length of segment : 357 time for calcul the mask position with numpy : 0.0012400150299072266 nb_pixel_total : 24066 time to create 1 rle with old method : 0.025964975357055664 length of segment : 225 time for calcul the mask position with numpy : 0.004365444183349609 nb_pixel_total : 98711 time to create 1 rle with old method : 0.10686969757080078 length of segment : 494 time for calcul the mask position with numpy : 0.0015232563018798828 nb_pixel_total : 19363 time to create 1 rle with old method : 0.02088761329650879 length of segment : 221 time for calcul the mask position with numpy : 0.002077817916870117 nb_pixel_total : 43603 time to create 1 rle with old method : 0.04695701599121094 length of segment : 341 time for calcul the mask position with numpy : 0.001172780990600586 nb_pixel_total : 12833 time to create 1 rle with old method : 0.016437530517578125 length of segment : 306 time for calcul the mask position with numpy : 0.0026617050170898438 nb_pixel_total : 64511 time to create 1 rle with old method : 0.07020759582519531 length of segment : 486 time for calcul the mask position with numpy : 0.00208282470703125 nb_pixel_total : 58091 time to create 1 rle with old method : 0.06467270851135254 length of segment : 306 time for calcul the mask position with numpy : 0.0006089210510253906 nb_pixel_total : 17595 time to create 1 rle with old method : 0.019505977630615234 length of segment : 252 time for calcul the mask position with numpy : 0.001096487045288086 nb_pixel_total : 16476 time to create 1 rle with old method : 0.01955437660217285 length of segment : 232 time for calcul the mask position with numpy : 0.0010831356048583984 nb_pixel_total : 28544 time to create 1 rle with old method : 0.031882286071777344 length of segment : 174 time for calcul the mask position with numpy : 0.0030884742736816406 nb_pixel_total : 110412 time to create 1 rle with old method : 0.12462615966796875 length of segment : 288 time for calcul the mask position with numpy : 0.0006225109100341797 nb_pixel_total : 12601 time to create 1 rle with old method : 0.013945579528808594 length of segment : 118 time for calcul the mask position with numpy : 0.002067089080810547 nb_pixel_total : 49566 time to create 1 rle with old method : 0.05418968200683594 length of segment : 369 time for calcul the mask position with numpy : 0.004001140594482422 nb_pixel_total : 82772 time to create 1 rle with old method : 0.08865547180175781 length of segment : 434 time for calcul the mask position with numpy : 0.006257534027099609 nb_pixel_total : 175863 time to create 1 rle with new method : 0.012271881103515625 length of segment : 604 time for calcul the mask position with numpy : 0.0014853477478027344 nb_pixel_total : 37984 time to create 1 rle with old method : 0.04176020622253418 length of segment : 266 time for calcul the mask position with numpy : 0.0009949207305908203 nb_pixel_total : 25145 time to create 1 rle with old method : 0.028098583221435547 length of segment : 208 time for calcul the mask position with numpy : 0.0002739429473876953 nb_pixel_total : 5259 time to create 1 rle with old method : 0.005994558334350586 length of segment : 95 time for calcul the mask position with numpy : 0.0004756450653076172 nb_pixel_total : 11820 time to create 1 rle with old method : 0.013345003128051758 length of segment : 110 time for calcul the mask position with numpy : 0.00041222572326660156 nb_pixel_total : 9408 time to create 1 rle with old method : 0.010736465454101562 length of segment : 132 time for calcul the mask position with numpy : 0.001743316650390625 nb_pixel_total : 30913 time to create 1 rle with old method : 0.03490948677062988 length of segment : 274 time for calcul the mask position with numpy : 0.0014917850494384766 nb_pixel_total : 42725 time to create 1 rle with old method : 0.045694828033447266 length of segment : 202 time for calcul the mask position with numpy : 0.0013761520385742188 nb_pixel_total : 45264 time to create 1 rle with old method : 0.04813408851623535 length of segment : 281 time for calcul the mask position with numpy : 0.00017309188842773438 nb_pixel_total : 5345 time to create 1 rle with old method : 0.006077766418457031 length of segment : 159 time for calcul the mask position with numpy : 0.0034613609313964844 nb_pixel_total : 126110 time to create 1 rle with old method : 0.13224458694458008 length of segment : 401 time for calcul the mask position with numpy : 0.0015690326690673828 nb_pixel_total : 46519 time to create 1 rle with old method : 0.05116891860961914 length of segment : 330 time for calcul the mask position with numpy : 0.001760244369506836 nb_pixel_total : 49916 time to create 1 rle with old method : 0.05350804328918457 length of segment : 317 time for calcul the mask position with numpy : 0.0001761913299560547 nb_pixel_total : 5508 time to create 1 rle with old method : 0.0064313411712646484 length of segment : 105 time for calcul the mask position with numpy : 0.000576019287109375 nb_pixel_total : 9686 time to create 1 rle with old method : 0.011331558227539062 length of segment : 299 time for calcul the mask position with numpy : 0.0006699562072753906 nb_pixel_total : 13623 time to create 1 rle with old method : 0.015288352966308594 length of segment : 146 time for calcul the mask position with numpy : 0.00025725364685058594 nb_pixel_total : 7884 time to create 1 rle with old method : 0.00961923599243164 length of segment : 108 time for calcul the mask position with numpy : 0.0010328292846679688 nb_pixel_total : 17891 time to create 1 rle with old method : 0.020307064056396484 length of segment : 151 time for calcul the mask position with numpy : 0.0012564659118652344 nb_pixel_total : 21850 time to create 1 rle with old method : 0.025766849517822266 length of segment : 213 time for calcul the mask position with numpy : 0.001190185546875 nb_pixel_total : 13661 time to create 1 rle with old method : 0.015342950820922852 length of segment : 145 time for calcul the mask position with numpy : 0.0010251998901367188 nb_pixel_total : 8534 time to create 1 rle with old method : 0.010535001754760742 length of segment : 149 time for calcul the mask position with numpy : 0.000438690185546875 nb_pixel_total : 7337 time to create 1 rle with old method : 0.008358240127563477 length of segment : 97 time for calcul the mask position with numpy : 0.0015079975128173828 nb_pixel_total : 31928 time to create 1 rle with old method : 0.03560638427734375 length of segment : 268 time for calcul the mask position with numpy : 0.0006618499755859375 nb_pixel_total : 13016 time to create 1 rle with old method : 0.014756441116333008 length of segment : 100 time for calcul the mask position with numpy : 0.0025200843811035156 nb_pixel_total : 52187 time to create 1 rle with old method : 0.057611703872680664 length of segment : 283 time for calcul the mask position with numpy : 0.0004558563232421875 nb_pixel_total : 9199 time to create 1 rle with old method : 0.010314702987670898 length of segment : 110 time for calcul the mask position with numpy : 0.0012073516845703125 nb_pixel_total : 16437 time to create 1 rle with old method : 0.019245624542236328 length of segment : 218 time for calcul the mask position with numpy : 0.0018155574798583984 nb_pixel_total : 31222 time to create 1 rle with old method : 0.03390073776245117 length of segment : 269 time for calcul the mask position with numpy : 0.0006856918334960938 nb_pixel_total : 9221 time to create 1 rle with old method : 0.01045536994934082 length of segment : 102 time for calcul the mask position with numpy : 0.0012135505676269531 nb_pixel_total : 24322 time to create 1 rle with old method : 0.026775836944580078 length of segment : 203 time for calcul the mask position with numpy : 0.0015680789947509766 nb_pixel_total : 32167 time to create 1 rle with old method : 0.035299062728881836 length of segment : 188 time for calcul the mask position with numpy : 0.0007476806640625 nb_pixel_total : 12849 time to create 1 rle with old method : 0.014470100402832031 length of segment : 101 time for calcul the mask position with numpy : 0.0011091232299804688 nb_pixel_total : 18446 time to create 1 rle with old method : 0.020514965057373047 length of segment : 223 time for calcul the mask position with numpy : 0.001535177230834961 nb_pixel_total : 30519 time to create 1 rle with old method : 0.03368687629699707 length of segment : 340 time for calcul the mask position with numpy : 0.0006194114685058594 nb_pixel_total : 10696 time to create 1 rle with old method : 0.011964082717895508 length of segment : 117 time for calcul the mask position with numpy : 0.0030264854431152344 nb_pixel_total : 34548 time to create 1 rle with old method : 0.03850674629211426 length of segment : 385 time for calcul the mask position with numpy : 0.0010895729064941406 nb_pixel_total : 11114 time to create 1 rle with old method : 0.012747526168823242 length of segment : 118 time for calcul the mask position with numpy : 0.0018453598022460938 nb_pixel_total : 22340 time to create 1 rle with old method : 0.025115251541137695 length of segment : 188 time for calcul the mask position with numpy : 0.0006461143493652344 nb_pixel_total : 9587 time to create 1 rle with old method : 0.011463165283203125 length of segment : 133 time for calcul the mask position with numpy : 0.00078582763671875 nb_pixel_total : 20133 time to create 1 rle with old method : 0.02250075340270996 length of segment : 149 time for calcul the mask position with numpy : 0.0002789497375488281 nb_pixel_total : 4707 time to create 1 rle with old method : 0.005589485168457031 length of segment : 102 time for calcul the mask position with numpy : 0.0007500648498535156 nb_pixel_total : 11311 time to create 1 rle with old method : 0.012979269027709961 length of segment : 119 time for calcul the mask position with numpy : 0.00048089027404785156 nb_pixel_total : 6119 time to create 1 rle with old method : 0.007435798645019531 length of segment : 108 time for calcul the mask position with numpy : 0.0004863739013671875 nb_pixel_total : 8343 time to create 1 rle with old method : 0.010378599166870117 length of segment : 92 time for calcul the mask position with numpy : 0.0007634162902832031 nb_pixel_total : 13822 time to create 1 rle with old method : 0.017039060592651367 length of segment : 157 time for calcul the mask position with numpy : 0.0014989376068115234 nb_pixel_total : 21561 time to create 1 rle with old method : 0.035314321517944336 length of segment : 241 time for calcul the mask position with numpy : 0.0003466606140136719 nb_pixel_total : 3770 time to create 1 rle with old method : 0.004358768463134766 length of segment : 95 time spent for convertir_results : 7.419930696487427 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 104 chid ids of type : 3594 Number RLEs to save : 24163 save missing photos in datou_result : time spend for datou_step_exec : 63.08922290802002 time spend to save output : 1.4061474800109863 total time spend for step 1 : 64.495370388031 step2:crop_condition Fri Feb 7 13:51:34 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 5 ! batch 1 Loaded 104 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 82 About to insert : list_path_to_insert length 82 new photo from crops ! About to upload 82 photos upload in portfolio : 3736932 init cache_photo without model_param we have 82 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738932712_3823264 we have uploaded 82 photos in the portfolio 3736932 time of upload the photos Elapsed time : 18.484238147735596 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 11 About to insert : list_path_to_insert length 11 new photo from crops ! About to upload 11 photos upload in portfolio : 3736932 init cache_photo without model_param we have 11 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738932733_3823264 we have uploaded 11 photos in the portfolio 3736932 time of upload the photos Elapsed time : 5.859003067016602 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 3736932 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738932739_3823264 we have uploaded 3 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.6939752101898193 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! About to upload 6 photos upload in portfolio : 3736932 init cache_photo without model_param we have 6 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738932742_3823264 we have uploaded 6 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.6428015232086182 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3736932 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738932745_3823264 we have uploaded 2 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.819258451461792 we have finished the crop for the class : pet_fonce delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1335495963, 1335495953, 1335495947, 1335495830, 1335495824] Looping around the photos to save general results len do output : 104 /1335519401Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519402Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519403Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519404Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519405Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519406Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519407Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519408Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519409Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519410Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519411Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519412Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519413Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519414Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519415Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519416Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519417Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519418Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519419Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519420Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519421Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519422Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519423Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519424Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519425Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519426Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519427Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519428Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519429Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519430Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519431Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519432Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519433Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519434Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519435Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519436Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519437Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519439Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519440Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519441Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519442Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519443Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519444Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519445Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519446Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519447Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519448Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519449Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519450Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519451Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519452Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519453Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519454Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519455Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519456Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519457Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519458Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519459Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519460Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519461Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519462Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519463Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519464Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519465Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519467Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519468Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519469Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519470Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519471Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519472Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519473Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519474Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519475Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519476Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519477Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519478Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519479Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519480Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519481Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519482Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519483Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519484Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519492Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519493Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519494Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519495Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519496Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519498Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519499Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519500Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519501Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519502Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519503Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519511Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519512Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519513Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519517Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519518Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519519Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519520Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519521Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519522Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519523Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335519524Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2564396') ('3318', '20334534', '1335495963', None, None, None, None, None, '2564396') ('3318', None, None, None, None, None, None, None, '2564396') ('3318', '20334534', '1335495953', None, None, None, None, None, '2564396') ('3318', None, None, None, None, None, None, None, '2564396') ('3318', '20334534', '1335495947', None, None, None, None, None, '2564396') ('3318', None, None, None, None, None, None, None, '2564396') ('3318', '20334534', '1335495830', None, None, None, None, None, '2564396') ('3318', None, None, None, None, None, None, None, '2564396') ('3318', '20334534', '1335495824', None, None, None, None, None, '2564396') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 317 time used for this insertion : 0.027326107025146484 save_final save missing photos in datou_result : time spend for datou_step_exec : 51.60629725456238 time spend to save output : 0.0312197208404541 total time spend for step 2 : 51.63751697540283 step3:rle_unique_nms_with_priority Fri Feb 7 13:52:26 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 104 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 17 nb_hashtags : 3 time to prepare the origin masks : 4.677464962005615 time for calcul the mask position with numpy : 0.3944392204284668 nb_pixel_total : 7680687 time to create 1 rle with new method : 1.356076955795288 time for calcul the mask position with numpy : 0.025538206100463867 nb_pixel_total : 6848 time to create 1 rle with old method : 0.007445096969604492 time for calcul the mask position with numpy : 0.026105642318725586 nb_pixel_total : 123797 time to create 1 rle with old method : 0.13498449325561523 time for calcul the mask position with numpy : 0.02563762664794922 nb_pixel_total : 13382 time to create 1 rle with old method : 0.016003847122192383 time for calcul the mask position with numpy : 0.025798797607421875 nb_pixel_total : 6498 time to create 1 rle with old method : 0.007457256317138672 time for calcul the mask position with numpy : 0.025795459747314453 nb_pixel_total : 37266 time to create 1 rle with old method : 0.040920257568359375 time for calcul the mask position with numpy : 0.03844428062438965 nb_pixel_total : 12739 time to create 1 rle with old method : 0.02002739906311035 time for calcul the mask position with numpy : 0.046450138092041016 nb_pixel_total : 30986 time to create 1 rle with old method : 0.035778045654296875 time for calcul the mask position with numpy : 0.041149139404296875 nb_pixel_total : 6925 time to create 1 rle with old method : 0.008816242218017578 time for calcul the mask position with numpy : 0.04218602180480957 nb_pixel_total : 182325 time to create 1 rle with new method : 0.6560094356536865 time for calcul the mask position with numpy : 0.040502071380615234 nb_pixel_total : 20937 time to create 1 rle with old method : 0.023040056228637695 time for calcul the mask position with numpy : 0.03391313552856445 nb_pixel_total : 11750 time to create 1 rle with old method : 0.012917518615722656 time for calcul the mask position with numpy : 0.025105953216552734 nb_pixel_total : 43278 time to create 1 rle with old method : 0.04544973373413086 time for calcul the mask position with numpy : 0.024867534637451172 nb_pixel_total : 19167 time to create 1 rle with old method : 0.02138352394104004 time for calcul the mask position with numpy : 0.02581334114074707 nb_pixel_total : 25750 time to create 1 rle with old method : 0.029279470443725586 time for calcul the mask position with numpy : 0.024948835372924805 nb_pixel_total : 21432 time to create 1 rle with old method : 0.02430868148803711 time for calcul the mask position with numpy : 0.026086807250976562 nb_pixel_total : 8998 time to create 1 rle with old method : 0.010357379913330078 time for calcul the mask position with numpy : 0.025705814361572266 nb_pixel_total : 41635 time to create 1 rle with old method : 0.04514646530151367 create new chi : 3.483691930770874 time to delete rle : 0.01970505714416504 batch 1 Loaded 35 chid ids of type : 3594 +++++++++++++++++++Number RLEs to save : 9168 TO DO : save crop sub photo not yet done ! save time : 0.5768172740936279 nb_obj : 20 nb_hashtags : 5 time to prepare the origin masks : 8.929470300674438 time for calcul the mask position with numpy : 0.9708805084228516 nb_pixel_total : 6949872 time to create 1 rle with new method : 1.2358956336975098 time for calcul the mask position with numpy : 0.02498602867126465 nb_pixel_total : 55273 time to create 1 rle with old method : 0.06047248840332031 time for calcul the mask position with numpy : 0.027393579483032227 nb_pixel_total : 30014 time to create 1 rle with old method : 0.0331268310546875 time for calcul the mask position with numpy : 0.02999401092529297 nb_pixel_total : 11095 time to create 1 rle with old method : 0.012807846069335938 time for calcul the mask position with numpy : 0.02746725082397461 nb_pixel_total : 19912 time to create 1 rle with old method : 0.029654979705810547 time for calcul the mask position with numpy : 0.031113862991333008 nb_pixel_total : 94805 time to create 1 rle with old method : 0.10492634773254395 time for calcul the mask position with numpy : 0.02602410316467285 nb_pixel_total : 24417 time to create 1 rle with old method : 0.025574922561645508 time for calcul the mask position with numpy : 0.02512812614440918 nb_pixel_total : 35920 time to create 1 rle with old method : 0.03695034980773926 time for calcul the mask position with numpy : 0.02507805824279785 nb_pixel_total : 28204 time to create 1 rle with old method : 0.030175209045410156 time for calcul the mask position with numpy : 0.02565908432006836 nb_pixel_total : 15847 time to create 1 rle with old method : 0.0175626277923584 time for calcul the mask position with numpy : 0.037804365158081055 nb_pixel_total : 8799 time to create 1 rle with old method : 0.009577274322509766 time for calcul the mask position with numpy : 0.03863239288330078 nb_pixel_total : 47934 time to create 1 rle with old method : 0.06668567657470703 time for calcul the mask position with numpy : 0.05433797836303711 nb_pixel_total : 398305 time to create 1 rle with new method : 0.7804481983184814 time for calcul the mask position with numpy : 0.047187089920043945 nb_pixel_total : 145487 time to create 1 rle with old method : 0.1531989574432373 time for calcul the mask position with numpy : 0.04423260688781738 nb_pixel_total : 15723 time to create 1 rle with old method : 0.01665782928466797 time for calcul the mask position with numpy : 0.040997982025146484 nb_pixel_total : 129857 time to create 1 rle with old method : 0.1380631923675537 time for calcul the mask position with numpy : 0.04139065742492676 nb_pixel_total : 56914 time to create 1 rle with old method : 0.0657951831817627 time for calcul the mask position with numpy : 0.042208194732666016 nb_pixel_total : 24799 time to create 1 rle with old method : 0.025829792022705078 time for calcul the mask position with numpy : 0.04009747505187988 nb_pixel_total : 5629 time to create 1 rle with old method : 0.006489992141723633 time for calcul the mask position with numpy : 0.039395809173583984 nb_pixel_total : 24773 time to create 1 rle with old method : 0.025502920150756836 time for calcul the mask position with numpy : 0.03973507881164551 nb_pixel_total : 170821 time to create 1 rle with new method : 0.6665644645690918 create new chi : 5.3207526206970215 time to delete rle : 0.0028765201568603516 batch 1 Loaded 42 chid ids of type : 3594 ++++++++++++++++++++++++Number RLEs to save : 13831 TO DO : save crop sub photo not yet done ! save time : 0.811976432800293 nb_obj : 14 nb_hashtags : 3 time to prepare the origin masks : 4.119173765182495 time for calcul the mask position with numpy : 0.42061853408813477 nb_pixel_total : 7776135 time to create 1 rle with new method : 0.6604359149932861 time for calcul the mask position with numpy : 0.024245262145996094 nb_pixel_total : 16476 time to create 1 rle with old method : 0.018035411834716797 time for calcul the mask position with numpy : 0.024088621139526367 nb_pixel_total : 17595 time to create 1 rle with old method : 0.018860340118408203 time for calcul the mask position with numpy : 0.02423858642578125 nb_pixel_total : 58091 time to create 1 rle with old method : 0.06421232223510742 time for calcul the mask position with numpy : 0.023774147033691406 nb_pixel_total : 64511 time to create 1 rle with old method : 0.07188105583190918 time for calcul the mask position with numpy : 0.0238797664642334 nb_pixel_total : 12833 time to create 1 rle with old method : 0.013647079467773438 time for calcul the mask position with numpy : 0.02622246742248535 nb_pixel_total : 43603 time to create 1 rle with old method : 0.06645393371582031 time for calcul the mask position with numpy : 0.025401592254638672 nb_pixel_total : 19363 time to create 1 rle with old method : 0.021955251693725586 time for calcul the mask position with numpy : 0.025327444076538086 nb_pixel_total : 98711 time to create 1 rle with old method : 0.10170578956604004 time for calcul the mask position with numpy : 0.02372884750366211 nb_pixel_total : 24066 time to create 1 rle with old method : 0.025470256805419922 time for calcul the mask position with numpy : 0.02458977699279785 nb_pixel_total : 65752 time to create 1 rle with old method : 0.06963324546813965 time for calcul the mask position with numpy : 0.0381622314453125 nb_pixel_total : 12515 time to create 1 rle with old method : 0.013365030288696289 time for calcul the mask position with numpy : 0.03949165344238281 nb_pixel_total : 18625 time to create 1 rle with old method : 0.02005290985107422 time for calcul the mask position with numpy : 0.04047513008117676 nb_pixel_total : 21921 time to create 1 rle with old method : 0.023703813552856445 time for calcul the mask position with numpy : 0.03908276557922363 nb_pixel_total : 44203 time to create 1 rle with old method : 0.0469050407409668 create new chi : 2.1064717769622803 time to delete rle : 0.0013170242309570312 batch 1 Loaded 29 chid ids of type : 3594 ++++++++++++++++++++++++Number RLEs to save : 9968 TO DO : save crop sub photo not yet done ! save time : 0.6091670989990234 nb_obj : 22 nb_hashtags : 4 time to prepare the origin masks : 6.149040937423706 time for calcul the mask position with numpy : 0.5296103954315186 nb_pixel_total : 7362246 time to create 1 rle with new method : 0.7429759502410889 time for calcul the mask position with numpy : 0.023818254470825195 nb_pixel_total : 7884 time to create 1 rle with old method : 0.008595466613769531 time for calcul the mask position with numpy : 0.023790836334228516 nb_pixel_total : 13623 time to create 1 rle with old method : 0.014786720275878906 time for calcul the mask position with numpy : 0.025587081909179688 nb_pixel_total : 9686 time to create 1 rle with old method : 0.010437250137329102 time for calcul the mask position with numpy : 0.02436351776123047 nb_pixel_total : 5508 time to create 1 rle with old method : 0.005872964859008789 time for calcul the mask position with numpy : 0.024982213973999023 nb_pixel_total : 49916 time to create 1 rle with old method : 0.05355072021484375 time for calcul the mask position with numpy : 0.023858070373535156 nb_pixel_total : 46519 time to create 1 rle with old method : 0.04928898811340332 time for calcul the mask position with numpy : 0.024733543395996094 nb_pixel_total : 126110 time to create 1 rle with old method : 0.13399720191955566 time for calcul the mask position with numpy : 0.02400994300842285 nb_pixel_total : 4632 time to create 1 rle with old method : 0.005254030227661133 time for calcul the mask position with numpy : 0.024940967559814453 nb_pixel_total : 45264 time to create 1 rle with old method : 0.06181788444519043 time for calcul the mask position with numpy : 0.02414703369140625 nb_pixel_total : 42725 time to create 1 rle with old method : 0.04415583610534668 time for calcul the mask position with numpy : 0.024007558822631836 nb_pixel_total : 30913 time to create 1 rle with old method : 0.03381061553955078 time for calcul the mask position with numpy : 0.02508854866027832 nb_pixel_total : 9408 time to create 1 rle with old method : 0.009746789932250977 time for calcul the mask position with numpy : 0.02524113655090332 nb_pixel_total : 11820 time to create 1 rle with old method : 0.012309789657592773 time for calcul the mask position with numpy : 0.02523660659790039 nb_pixel_total : 5259 time to create 1 rle with old method : 0.005402088165283203 time for calcul the mask position with numpy : 0.025933265686035156 nb_pixel_total : 25145 time to create 1 rle with old method : 0.026401281356811523 time for calcul the mask position with numpy : 0.02379465103149414 nb_pixel_total : 37984 time to create 1 rle with old method : 0.05288052558898926 time for calcul the mask position with numpy : 0.02916574478149414 nb_pixel_total : 175863 time to create 1 rle with new method : 0.6126062870025635 time for calcul the mask position with numpy : 0.027224302291870117 nb_pixel_total : 82772 time to create 1 rle with old method : 0.08236289024353027 time for calcul the mask position with numpy : 0.025162458419799805 nb_pixel_total : 49566 time to create 1 rle with old method : 0.051789045333862305 time for calcul the mask position with numpy : 0.02380824089050293 nb_pixel_total : 12601 time to create 1 rle with old method : 0.013654947280883789 time for calcul the mask position with numpy : 0.025480031967163086 nb_pixel_total : 110412 time to create 1 rle with old method : 0.10980772972106934 time for calcul the mask position with numpy : 0.024127483367919922 nb_pixel_total : 28544 time to create 1 rle with old method : 0.0286867618560791 create new chi : 3.315093755722046 time to delete rle : 0.0016067028045654297 batch 1 Loaded 45 chid ids of type : 3594 +++++++++++++++++++++++++++++++Number RLEs to save : 12917 TO DO : save crop sub photo not yet done ! save time : 0.7386472225189209 nb_obj : 30 nb_hashtags : 3 time to prepare the origin masks : 3.847456932067871 time for calcul the mask position with numpy : 0.09200310707092285 nb_pixel_total : 7765979 time to create 1 rle with new method : 0.18947792053222656 time for calcul the mask position with numpy : 0.032341718673706055 nb_pixel_total : 18446 time to create 1 rle with old method : 0.01910257339477539 time for calcul the mask position with numpy : 0.03143143653869629 nb_pixel_total : 6119 time to create 1 rle with old method : 0.006745338439941406 time for calcul the mask position with numpy : 0.03213238716125488 nb_pixel_total : 9587 time to create 1 rle with old method : 0.010803699493408203 time for calcul the mask position with numpy : 0.03219246864318848 nb_pixel_total : 10696 time to create 1 rle with old method : 0.011057615280151367 time for calcul the mask position with numpy : 0.03245949745178223 nb_pixel_total : 9221 time to create 1 rle with old method : 0.010074615478515625 time for calcul the mask position with numpy : 0.03268098831176758 nb_pixel_total : 31222 time to create 1 rle with old method : 0.04846978187561035 time for calcul the mask position with numpy : 0.03829193115234375 nb_pixel_total : 52187 time to create 1 rle with old method : 0.05456995964050293 time for calcul the mask position with numpy : 0.03298664093017578 nb_pixel_total : 21561 time to create 1 rle with old method : 0.023436307907104492 time for calcul the mask position with numpy : 0.031867265701293945 nb_pixel_total : 16437 time to create 1 rle with old method : 0.01680135726928711 time for calcul the mask position with numpy : 0.033329010009765625 nb_pixel_total : 4707 time to create 1 rle with old method : 0.005265951156616211 time for calcul the mask position with numpy : 0.03328728675842285 nb_pixel_total : 7337 time to create 1 rle with old method : 0.008002519607543945 time for calcul the mask position with numpy : 0.032701730728149414 nb_pixel_total : 8534 time to create 1 rle with old method : 0.009305715560913086 time for calcul the mask position with numpy : 0.032483577728271484 nb_pixel_total : 8812 time to create 1 rle with old method : 0.0097198486328125 time for calcul the mask position with numpy : 0.03228592872619629 nb_pixel_total : 22340 time to create 1 rle with old method : 0.024286508560180664 time for calcul the mask position with numpy : 0.03275656700134277 nb_pixel_total : 30519 time to create 1 rle with old method : 0.03234434127807617 time for calcul the mask position with numpy : 0.031350135803222656 nb_pixel_total : 17891 time to create 1 rle with old method : 0.019219636917114258 time for calcul the mask position with numpy : 0.03313922882080078 nb_pixel_total : 21850 time to create 1 rle with old method : 0.02270960807800293 time for calcul the mask position with numpy : 0.032019853591918945 nb_pixel_total : 24322 time to create 1 rle with old method : 0.02365732192993164 time for calcul the mask position with numpy : 0.03144121170043945 nb_pixel_total : 8343 time to create 1 rle with old method : 0.009065628051757812 time for calcul the mask position with numpy : 0.032918691635131836 nb_pixel_total : 11311 time to create 1 rle with old method : 0.011905908584594727 time for calcul the mask position with numpy : 0.03317761421203613 nb_pixel_total : 13661 time to create 1 rle with old method : 0.013914346694946289 time for calcul the mask position with numpy : 0.03167867660522461 nb_pixel_total : 11114 time to create 1 rle with old method : 0.011211395263671875 time for calcul the mask position with numpy : 0.031572580337524414 nb_pixel_total : 31928 time to create 1 rle with old method : 0.032331228256225586 time for calcul the mask position with numpy : 0.033846378326416016 nb_pixel_total : 34548 time to create 1 rle with old method : 0.03559446334838867 time for calcul the mask position with numpy : 0.03366351127624512 nb_pixel_total : 3770 time to create 1 rle with old method : 0.00439906120300293 time for calcul the mask position with numpy : 0.03453469276428223 nb_pixel_total : 32167 time to create 1 rle with old method : 0.039885520935058594 time for calcul the mask position with numpy : 0.03425884246826172 nb_pixel_total : 20133 time to create 1 rle with old method : 0.022549152374267578 time for calcul the mask position with numpy : 0.03449440002441406 nb_pixel_total : 13793 time to create 1 rle with old method : 0.015015602111816406 time for calcul the mask position with numpy : 0.03498339653015137 nb_pixel_total : 13016 time to create 1 rle with old method : 0.01494908332824707 time for calcul the mask position with numpy : 0.03445291519165039 nb_pixel_total : 12849 time to create 1 rle with old method : 0.014138221740722656 create new chi : 1.872706413269043 time to delete rle : 0.0014727115631103516 batch 1 Loaded 61 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++Number RLEs to save : 12424 TO DO : save crop sub photo not yet done ! save time : 0.7313201427459717 map_output_result : {1335495963: (0.0, 'Should be the crop_list due to order', 0.0), 1335495953: (0.0, 'Should be the crop_list due to order', 0.0), 1335495947: (0.0, 'Should be the crop_list due to order', 0.0), 1335495830: (0.0, 'Should be the crop_list due to order', 0.0), 1335495824: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1335495963, 1335495953, 1335495947, 1335495830, 1335495824] Looping around the photos to save general results len do output : 5 /1335495963.Didn't retrieve data . /1335495953.Didn't retrieve data . /1335495947.Didn't retrieve data . /1335495830.Didn't retrieve data . /1335495824.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2564396') ('3318', '20334534', '1335495963', None, None, None, None, None, '2564396') ('3318', None, None, None, None, None, None, None, '2564396') ('3318', '20334534', '1335495953', None, None, None, None, None, '2564396') ('3318', None, None, None, None, None, None, None, '2564396') ('3318', '20334534', '1335495947', None, None, None, None, None, '2564396') ('3318', None, None, None, None, None, None, None, '2564396') ('3318', '20334534', '1335495830', None, None, None, None, None, '2564396') ('3318', None, None, None, None, None, None, None, '2564396') ('3318', '20334534', '1335495824', None, None, None, None, None, '2564396') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 15 time used for this insertion : 0.012077808380126953 save_final save missing photos in datou_result : time spend for datou_step_exec : 48.15807867050171 time spend to save output : 0.012434720993041992 total time spend for step 3 : 48.17051339149475 step4:ventilate_hashtags_in_portfolio Fri Feb 7 13:53:14 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 20334534 get user id for portfolio 20334534 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20334534 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_clair','carton','mal_croppe','environnement','autre','background','flou','papier','pehd','pet_fonce','metal')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20334534 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_clair','carton','mal_croppe','environnement','autre','background','flou','papier','pehd','pet_fonce','metal')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20334534 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_clair','carton','mal_croppe','environnement','autre','background','flou','papier','pehd','pet_fonce','metal')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/20335032,20335033,20335034,20335035,20335036,20335037,20335038,20335039,20335040,20335041,20335042?tags=pet_clair,carton,mal_croppe,environnement,autre,background,flou,papier,pehd,pet_fonce,metal Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1335495963, 1335495953, 1335495947, 1335495830, 1335495824] Looping around the photos to save general results len do output : 1 /20334534. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2564396') ('3318', '20334534', '1335495963', None, None, None, None, None, '2564396') ('3318', None, None, None, None, None, None, None, '2564396') ('3318', '20334534', '1335495953', None, None, None, None, None, '2564396') ('3318', None, None, None, None, None, None, None, '2564396') ('3318', '20334534', '1335495947', None, None, None, None, None, '2564396') ('3318', None, None, None, None, None, None, None, '2564396') ('3318', '20334534', '1335495830', None, None, None, None, None, '2564396') ('3318', None, None, None, None, None, None, None, '2564396') ('3318', '20334534', '1335495824', None, None, None, None, None, '2564396') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.013839483261108398 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.658616065979004 time spend to save output : 0.014106988906860352 total time spend for step 4 : 1.6727230548858643 step5:final Fri Feb 7 13:53:16 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1335495963: ('0.09493347318672837',), 1335495953: ('0.09493347318672837',), 1335495947: ('0.09493347318672837',), 1335495830: ('0.09493347318672837',), 1335495824: ('0.09493347318672837',)} new output for save of step final : {1335495963: ('0.09493347318672837',), 1335495953: ('0.09493347318672837',), 1335495947: ('0.09493347318672837',), 1335495830: ('0.09493347318672837',), 1335495824: ('0.09493347318672837',)} [1335495963, 1335495953, 1335495947, 1335495830, 1335495824] Looping around the photos to save general results len do output : 5 /1335495963.Didn't retrieve data . /1335495953.Didn't retrieve data . /1335495947.Didn't retrieve data . /1335495830.Didn't retrieve data . /1335495824.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2564396') ('3318', '20334534', '1335495963', None, None, None, None, None, '2564396') ('3318', None, None, None, None, None, None, None, '2564396') ('3318', '20334534', '1335495953', None, None, None, None, None, '2564396') ('3318', None, None, None, None, None, None, None, '2564396') ('3318', '20334534', '1335495947', None, None, None, None, None, '2564396') ('3318', None, None, None, None, None, None, None, '2564396') ('3318', '20334534', '1335495830', None, None, None, None, None, '2564396') ('3318', None, None, None, None, None, None, None, '2564396') ('3318', '20334534', '1335495824', None, None, None, None, None, '2564396') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 15 time used for this insertion : 0.012293815612792969 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.10268640518188477 time spend to save output : 0.01262664794921875 total time spend for step 5 : 0.11531305313110352 step6:blur_detection Fri Feb 7 13:53:16 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4.jpg resize: (2160, 3840) 1335495963 -6.040959904479833 treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1.jpg resize: (2160, 3840) 1335495953 -6.194800644947648 treat image : temp/1738932629_3823264_1335495947_39828a529bbcc4ea25a3422187295da8.jpg resize: (2160, 3840) 1335495947 -6.099209069809032 treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c.jpg resize: (2160, 3840) 1335495830 -6.17455012103452 treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4.jpg resize: (2160, 3840) 1335495824 -6.138845493151763 treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837530_0.png resize: (159, 106) 1335519401 -2.868908116524825 treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837527_0.png resize: (219, 201) 1335519402 -3.0912087807652013 treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837525_0.png resize: (123, 91) 1335519403 -2.547885926575224 treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837539_0.png resize: (412, 411) 1335519404 -3.08628901672148 treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837526_0.png resize: (167, 172) 1335519405 -3.9794887290971244 treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837535_0.png resize: (194, 128) 1335519406 -1.1433933547415853 treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837536_0.png resize: (244, 219) 1335519407 -3.6967625073532044 treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837529_0.png resize: (206, 321) 1335519408 -3.7598480500131894 treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837528_0.png resize: (199, 130) 1335519409 -3.3134478355908894 treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837531_0.png resize: (161, 208) 1335519410 -3.7369810182698213 treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837532_0.png resize: (552, 529) 1335519411 -3.101609134815354 treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837540_0.png resize: (95, 99) 1335519412 -3.7737723147157514 treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837551_0.png resize: (254, 274) 1335519413 -4.354649355833735 treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837549_0.png resize: (873, 706) 1335519414 -5.102561875428162 treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837552_0.png resize: (101, 107) 1335519415 0.2630913429662697 treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837545_0.png resize: (194, 429) 1335519416 -3.94741498417341 treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837560_0.png resize: (195, 190) 1335519417 -4.141500617953905 treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837550_0.png resize: (200, 155) 1335519418 -3.9853822748665446 treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837546_0.png resize: (383, 490) 1335519419 -3.8719017294195535 treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837553_0.png resize: (239, 99) 1335519420 -3.2498733415865195 treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837548_0.png resize: (398, 470) 1335519421 -1.624323419965446 treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837541_0.png resize: (514, 427) 1335519422 -3.85461426584541 treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837556_0.png resize: (125, 239) 1335519423 -4.142876275518207 treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837543_0.png resize: (87, 94) 1335519424 -2.991034555325286 treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837554_0.png resize: (176, 261) 1335519425 -4.220891532054261 treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837542_0.png resize: (205, 194) 1335519426 -2.7310596265918163 treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837547_0.png resize: (225, 80) 1335519427 -2.2073889695794215 treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837559_0.png resize: (171, 144) 1335519428 -3.958738948802697 treat image : temp/1738932629_3823264_1335495947_39828a529bbcc4ea25a3422187295da8_rle_crop_3659837563_0.png resize: (109, 284) 1335519429 -2.435822323471196 treat image : temp/1738932629_3823264_1335495947_39828a529bbcc4ea25a3422187295da8_rle_crop_3659837575_0.png resize: (198, 151) 1335519430 -2.6673458103907817 treat image : temp/1738932629_3823264_1335495947_39828a529bbcc4ea25a3422187295da8_rle_crop_3659837569_0.png resize: (221, 195) 1335519431 -3.5955523881568623 treat image : temp/1738932629_3823264_1335495947_39828a529bbcc4ea25a3422187295da8_rle_crop_3659837564_0.png resize: (154, 169) 1335519432 -2.584025276056712 treat image : temp/1738932629_3823264_1335495947_39828a529bbcc4ea25a3422187295da8_rle_crop_3659837562_0.png resize: (290, 231) 1335519433 -4.154867279374025 treat image : temp/1738932629_3823264_1335495947_39828a529bbcc4ea25a3422187295da8_rle_crop_3659837568_0.png resize: (424, 368) 1335519434 -4.043171941537942 treat image : temp/1738932629_3823264_1335495947_39828a529bbcc4ea25a3422187295da8_rle_crop_3659837570_0.png resize: (192, 334) 1335519435 -4.025110747945171 treat image : temp/1738932629_3823264_1335495947_39828a529bbcc4ea25a3422187295da8_rle_crop_3659837572_0.png resize: (354, 255) 1335519436 -4.133414585968528 treat image : temp/1738932629_3823264_1335495947_39828a529bbcc4ea25a3422187295da8_rle_crop_3659837565_0.png resize: (122, 124) 1335519437 -1.4018038585419907 treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837587_0.png resize: (221, 255) 1335519439 -2.7601541196998163 treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837591_0.png resize: (357, 421) 1335519440 -3.7579767457980076 treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837588_0.png resize: (199, 317) 1335519441 -2.4424444729966592 treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837584_0.png resize: (95, 70) 1335519442 -2.6023456829162166 treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837597_0.png resize: (70, 160) 1335519443 -2.399473790596136 treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837595_0.png resize: (139, 158) 1335519444 -3.453590545609495 treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837585_0.png resize: (110, 132) 1335519445 -2.947302375237703 treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837581_0.png resize: (549, 485) 1335519446 -3.2051790926791055 treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837576_0.png resize: (173, 256) 1335519447 -1.4915865966713064 treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837589_0.png resize: (249, 281) 1335519448 -4.415192297289869 treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837583_0.png resize: (208, 189) 1335519449 -3.275066948924977 treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837577_0.png resize: (270, 445) 1335519450 -4.33255355421434 treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837582_0.png resize: (202, 267) 1335519451 -3.791538634853605 treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837590_0.png resize: (108, 90) 1335519452 -3.979389772632862 treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837586_0.png resize: (131, 100) 1335519453 -3.6505411685646063 treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837580_0.png resize: (433, 263) 1335519454 -3.698837096605952 treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837579_0.png resize: (367, 199) 1335519455 -3.8261465398171475 treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837592_0.png resize: (228, 316) 1335519456 -3.38875063785062 treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837620_0.png resize: (149, 181) 1335519457 0.29426505491293453 treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837604_0.png resize: (99, 170) 1335519458 -2.299060151498197 treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837625_0.png resize: (118, 179) 1335519459 -2.929614275047066 treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837613_0.png resize: (223, 110) 1335519460 -3.392083420315289 treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837599_0.png resize: (211, 137) 1335519461 -1.3606932848013489 treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837610_0.png resize: (199, 157) 1335519462 -3.4070425479998576 treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837621_0.png resize: (101, 69) 1335519463 -2.7988019247352307 treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837605_0.png resize: (281, 262) 1335519464 -1.8801790944144259 treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837602_0.png resize: (96, 95) 1335519465 -3.032245983884854 treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837612_0.png resize: (101, 156) 1335519467 -3.0475175462333115 treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837603_0.png resize: (223, 204) 1335519468 -4.064965978028678 treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837618_0.png resize: (185, 228) 1335519469 -3.1232513062221847 treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837607_0.png resize: (146, 181) 1335519470 -3.1871203019242746 treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837600_0.png resize: (136, 192) 1335519471 -3.001682614862515 treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837627_0.png resize: (89, 64) 1335519472 -2.8405930660742773 treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837606_0.png resize: (108, 106) 1335519473 -1.0845566297460052 treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837601_0.png resize: (145, 166) 1335519474 -2.578881732286857 treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837623_0.png resize: (98, 84) 1335519475 -0.8767463883536821 treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837611_0.png resize: (186, 219) 1335519476 -3.1800334853275984 treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837614_0.png resize: (340, 110) 1335519477 -3.377457210506733 treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837619_0.png resize: (133, 89) 1335519478 -2.546420077591816 treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837626_0.png resize: (240, 121) 1335519479 -3.5087651229453374 treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837598_0.png resize: (149, 149) 1335519480 -2.063526205267885 treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837617_0.png resize: (117, 154) 1335519481 -4.352947793180556 treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837624_0.png resize: (90, 123) 1335519482 -2.7015146498709295 treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837608_0.png resize: (263, 161) 1335519483 -4.05626259331803 treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837616_0.png resize: (276, 317) 1335519484 -3.34146626856483 treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837533_0.png resize: (91, 99) 1335519492 -4.008709901676245 treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837534_0.png resize: (220, 202) 1335519493 -4.31759904059968 treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837524_0.png resize: (194, 260) 1335519494 -4.0573087448544785 treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837537_0.png resize: (53, 159) 1335519495 -2.6578427314012134 treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837561_0.png resize: (293, 312) 1335519496 -4.235169505133094 treat image : temp/1738932629_3823264_1335495947_39828a529bbcc4ea25a3422187295da8_rle_crop_3659837566_0.png resize: (355, 224) 1335519498 -3.345460648106693 treat image : temp/1738932629_3823264_1335495947_39828a529bbcc4ea25a3422187295da8_rle_crop_3659837573_0.png resize: (272, 280) 1335519499 -3.5223172624668395 treat image : temp/1738932629_3823264_1335495947_39828a529bbcc4ea25a3422187295da8_rle_crop_3659837571_0.png resize: (227, 91) 1335519500 -3.270908437270123 treat image : temp/1738932629_3823264_1335495947_39828a529bbcc4ea25a3422187295da8_rle_crop_3659837574_0.png resize: (252, 85) 1335519501 -2.4985737815685303 treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837596_0.png resize: (145, 138) 1335519502 -3.3162996208574342 treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837609_0.png resize: (102, 130) 1335519503 -3.0003724005890624 treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837555_0.png resize: (286, 242) 1335519511 -3.457383862607883 treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837593_0.png resize: (314, 215) 1335519512 -4.398607463482477 treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837594_0.png resize: (104, 80) 1335519513 -3.9802401928346387 treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837538_0.png resize: (134, 122) 1335519517 -3.290022760147009 treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837558_0.png resize: (152, 166) 1335519518 -2.558880769505655 treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837544_0.png resize: (231, 160) 1335519519 -3.6828831647980667 treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837578_0.png resize: (118, 150) 1335519520 -3.3806810776161345 treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837622_0.png resize: (111, 169) 1335519521 -3.6170638599718434 treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837615_0.png resize: (117, 113) 1335519522 -1.946814919465203 treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837557_0.png resize: (338, 397) 1335519523 -4.602300399650981 treat image : temp/1738932629_3823264_1335495947_39828a529bbcc4ea25a3422187295da8_rle_crop_3659837567_0.png resize: (225, 147) 1335519524 -3.385847447644859 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 109 time used for this insertion : 0.01772761344909668 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 109 time used for this insertion : 0.028927087783813477 save missing photos in datou_result : time spend for datou_step_exec : 19.155226469039917 time spend to save output : 0.05205845832824707 total time spend for step 6 : 19.207284927368164 step7:brightness Fri Feb 7 13:53:35 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4.jpg treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1.jpg treat image : temp/1738932629_3823264_1335495947_39828a529bbcc4ea25a3422187295da8.jpg treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c.jpg treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4.jpg treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837530_0.png treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837527_0.png treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837525_0.png treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837539_0.png treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837526_0.png treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837535_0.png treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837536_0.png treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837529_0.png treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837528_0.png treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837531_0.png treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837532_0.png treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837540_0.png treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837551_0.png treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837549_0.png treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837552_0.png treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837545_0.png treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837560_0.png treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837550_0.png treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837546_0.png treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837553_0.png treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837548_0.png treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837541_0.png treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837556_0.png treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837543_0.png treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837554_0.png treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837542_0.png treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837547_0.png treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837559_0.png treat image : temp/1738932629_3823264_1335495947_39828a529bbcc4ea25a3422187295da8_rle_crop_3659837563_0.png treat image : temp/1738932629_3823264_1335495947_39828a529bbcc4ea25a3422187295da8_rle_crop_3659837575_0.png treat image : temp/1738932629_3823264_1335495947_39828a529bbcc4ea25a3422187295da8_rle_crop_3659837569_0.png treat image : temp/1738932629_3823264_1335495947_39828a529bbcc4ea25a3422187295da8_rle_crop_3659837564_0.png treat image : temp/1738932629_3823264_1335495947_39828a529bbcc4ea25a3422187295da8_rle_crop_3659837562_0.png treat image : temp/1738932629_3823264_1335495947_39828a529bbcc4ea25a3422187295da8_rle_crop_3659837568_0.png treat image : temp/1738932629_3823264_1335495947_39828a529bbcc4ea25a3422187295da8_rle_crop_3659837570_0.png treat image : temp/1738932629_3823264_1335495947_39828a529bbcc4ea25a3422187295da8_rle_crop_3659837572_0.png treat image : temp/1738932629_3823264_1335495947_39828a529bbcc4ea25a3422187295da8_rle_crop_3659837565_0.png treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837587_0.png treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837591_0.png treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837588_0.png treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837584_0.png treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837597_0.png treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837595_0.png treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837585_0.png treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837581_0.png treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837576_0.png treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837589_0.png treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837583_0.png treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837577_0.png treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837582_0.png treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837590_0.png treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837586_0.png treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837580_0.png treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837579_0.png treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837592_0.png treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837620_0.png treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837604_0.png treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837625_0.png treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837613_0.png treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837599_0.png treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837610_0.png treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837621_0.png treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837605_0.png treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837602_0.png treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837612_0.png treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837603_0.png treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837618_0.png treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837607_0.png treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837600_0.png treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837627_0.png treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837606_0.png treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837601_0.png treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837623_0.png treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837611_0.png treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837614_0.png treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837619_0.png treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837626_0.png treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837598_0.png treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837617_0.png treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837624_0.png treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837608_0.png treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837616_0.png treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837533_0.png treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837534_0.png treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837524_0.png treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837537_0.png treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837561_0.png treat image : temp/1738932629_3823264_1335495947_39828a529bbcc4ea25a3422187295da8_rle_crop_3659837566_0.png treat image : temp/1738932629_3823264_1335495947_39828a529bbcc4ea25a3422187295da8_rle_crop_3659837573_0.png treat image : temp/1738932629_3823264_1335495947_39828a529bbcc4ea25a3422187295da8_rle_crop_3659837571_0.png treat image : temp/1738932629_3823264_1335495947_39828a529bbcc4ea25a3422187295da8_rle_crop_3659837574_0.png treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837596_0.png treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837609_0.png treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837555_0.png treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837593_0.png treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837594_0.png treat image : temp/1738932629_3823264_1335495963_52cdf001d0918a82f4ff81618f0e1ba4_rle_crop_3659837538_0.png treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837558_0.png treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837544_0.png treat image : temp/1738932629_3823264_1335495830_d55421cb938b2df4dd62769f41039c9c_rle_crop_3659837578_0.png treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837622_0.png treat image : temp/1738932629_3823264_1335495824_f3465845b4711b1f9303f1b946d410f4_rle_crop_3659837615_0.png treat image : temp/1738932629_3823264_1335495953_7e317aa9c88ef740968205fa817c11a1_rle_crop_3659837557_0.png treat image : temp/1738932629_3823264_1335495947_39828a529bbcc4ea25a3422187295da8_rle_crop_3659837567_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 109 time used for this insertion : 0.01534271240234375 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 109 time used for this insertion : 0.032990455627441406 save missing photos in datou_result : time spend for datou_step_exec : 5.544978618621826 time spend to save output : 0.05316305160522461 total time spend for step 7 : 5.598141670227051 step8:velours_tree Fri Feb 7 13:53:40 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.0912027359008789 time spend to save output : 3.24249267578125e-05 total time spend for step 8 : 0.09123516082763672 step9:send_mail_cod Fri Feb 7 13:53:41 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P20334534_07-02-2025_13_53_41.pdf 20335032 change filename to text .change filename to text .change filename to text .imagette203350321738932821 20335033 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette203350331738932821 20335034 imagette203350341738932822 20335036 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette203350361738932822 20335037 imagette203350371738932822 20335038 imagette203350381738932822 20335039 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette203350391738932822 20335040 imagette203350401738932823 20335041 change filename to text .change filename to text .imagette203350411738932823 20335042 imagette203350421738932823 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20334534 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20335032,20335033,20335034,20335035,20335036,20335037,20335038,20335039,20335040,20335041,20335042?tags=pet_clair,carton,mal_croppe,environnement,autre,background,flou,papier,pehd,pet_fonce,metal args[1335495963] : ((1335495963, -6.040959904479833, 492609224), (1335495963, -0.13186601258175842, 496442774), '0.09493347318672837') apple ((1335495963, -6.040959904479833, 492609224), (1335495963, -0.13186601258175842, 496442774), '0.09493347318672837') We are sending mail with results at report@fotonower.com args[1335495953] : ((1335495953, -6.194800644947648, 492609224), (1335495953, 0.0572956331035828, 2107752395), '0.09493347318672837') apple ((1335495953, -6.194800644947648, 492609224), (1335495953, 0.0572956331035828, 2107752395), '0.09493347318672837') We are sending mail with results at report@fotonower.com args[1335495947] : ((1335495947, -6.099209069809032, 492609224), (1335495947, -0.02794314062577633, 2107752395), '0.09493347318672837') apple ((1335495947, -6.099209069809032, 492609224), (1335495947, -0.02794314062577633, 2107752395), '0.09493347318672837') We are sending mail with results at report@fotonower.com args[1335495830] : ((1335495830, -6.17455012103452, 492609224), (1335495830, 0.07830184884642967, 2107752395), '0.09493347318672837') apple ((1335495830, -6.17455012103452, 492609224), (1335495830, 0.07830184884642967, 2107752395), '0.09493347318672837') We are sending mail with results at report@fotonower.com args[1335495824] : ((1335495824, -6.138845493151763, 492609224), (1335495824, 0.005285288378867049, 2107752395), '0.09493347318672837') apple ((1335495824, -6.138845493151763, 492609224), (1335495824, 0.005285288378867049, 2107752395), '0.09493347318672837') We are sending mail with results at report@fotonower.com refus_total : 0.09493347318672837 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20334534 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20334534_07-02-2025_13_53_41.pdf results_Auto_P20334534_07-02-2025_13_53_41.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20334534_07-02-2025_13_53_41.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','20334534','results_Auto_P20334534_07-02-2025_13_53_41.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20334534_07-02-2025_13_53_41.pdf','pdf','','0.41','0.09493347318672837') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/20334534

https://www.fotonower.com/image?json=false&list_photos_id=1335495963
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1335495953
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1335495947
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1335495830
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1335495824
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 9.49%
Veuillez trouver les photos des contaminants.

exemples de contaminants: pet_clair: https://www.fotonower.com/view/20335032?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/20335033?limit=200
exemples de contaminants: autre: https://www.fotonower.com/view/20335036?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/20335039?limit=200
exemples de contaminants: pet_fonce: https://www.fotonower.com/view/20335041?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20334534_07-02-2025_13_53_41.pdf.

Lien vers velours :https://www.fotonower.com/velours/20335032,20335033,20335034,20335035,20335036,20335037,20335038,20335039,20335040,20335041,20335042?tags=pet_clair,carton,mal_croppe,environnement,autre,background,flou,papier,pehd,pet_fonce,metal.


L'équipe Fotonower 202 b'' Server: nginx Date: Fri, 07 Feb 2025 12:53:45 GMT Content-Length: 0 Connection: close X-Message-Id: Dx7oPci2RyC9HC0iDp2SSw Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1335495963, 1335495953, 1335495947, 1335495830, 1335495824] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2564396') ('3318', '20334534', '1335495963', None, None, None, None, None, '2564396') ('3318', None, None, None, None, None, None, None, '2564396') ('3318', '20334534', '1335495953', None, None, None, None, None, '2564396') ('3318', None, None, None, None, None, None, None, '2564396') ('3318', '20334534', '1335495947', None, None, None, None, None, '2564396') ('3318', None, None, None, None, None, None, None, '2564396') ('3318', '20334534', '1335495830', None, None, None, None, None, '2564396') ('3318', None, None, None, None, None, None, None, '2564396') ('3318', '20334534', '1335495824', None, None, None, None, None, '2564396') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 5 time used for this insertion : 0.014891386032104492 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.726362466812134 time spend to save output : 0.015089273452758789 total time spend for step 9 : 4.741451740264893 step10:split_time_score Fri Feb 7 13:53:45 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('11', 5),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 07022025 20334534 Nombre de photos uploadées : 5 / 23040 (0%) 07022025 20334534 Nombre de photos taguées (types de déchets): 0 / 5 (0%) 07022025 20334534 Nombre de photos taguées (volume) : 0 / 5 (0%) elapsed_time : load_data_split_time_score 4.5299530029296875e-06 elapsed_time : order_list_meta_photo_and_scores 1.1682510375976562e-05 ????? elapsed_time : fill_and_build_computed_from_old_data 0.00041866302490234375 elapsed_time : insert_dashboard_record_day_entry 0.023506641387939453 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.03747212369216577 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20327792_07-02-2025_09_59_20.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20327792 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20327792 AND mptpi.`type`=3726 To do Qualite : 0.03574358804012652 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20323458_07-02-2025_07_55_35.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20323458 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20323458 AND mptpi.`type`=3726 To do Qualite : 0.02106041322187774 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20324555_07-02-2025_08_15_41.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20324555 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20324555 AND mptpi.`type`=3726 To do Qualite : 0.03036954721141696 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20327795_07-02-2025_09_56_11.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20327795 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20327795 AND mptpi.`type`=3726 To do Qualite : 0.09493347318672837 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20334534_07-02-2025_13_53_41.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20334534 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20334534 AND mptpi.`type`=3594 To do Qualite : 0.09868784102182541 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20334537_07-02-2025_13_44_42.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20334537 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20334537 AND mptpi.`type`=3594 To do Qualite : 0.021276444971890584 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20334538_07-02-2025_13_43_46.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20334538 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20334538 AND mptpi.`type`=3726 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'07022025': {'nb_upload': 5, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1335495963, 1335495953, 1335495947, 1335495830, 1335495824] Looping around the photos to save general results len do output : 1 /20334534Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2564396') ('3318', '20334534', '1335495963', None, None, None, None, None, '2564396') ('3318', None, None, None, None, None, None, None, '2564396') ('3318', '20334534', '1335495953', None, None, None, None, None, '2564396') ('3318', None, None, None, None, None, None, None, '2564396') ('3318', '20334534', '1335495947', None, None, None, None, None, '2564396') ('3318', None, None, None, None, None, None, None, '2564396') ('3318', '20334534', '1335495830', None, None, None, None, None, '2564396') ('3318', None, None, None, None, None, None, None, '2564396') ('3318', '20334534', '1335495824', None, None, None, None, None, '2564396') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.013166427612304688 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.6095428466796875 time spend to save output : 0.01343846321105957 total time spend for step 10 : 0.6229813098907471 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 5 set_done_treatment 92.95user 70.32system 3:20.74elapsed 81%CPU (0avgtext+0avgdata 5199268maxresident)k 531080inputs+70552outputs (2108major+9754369minor)pagefaults 0swaps