python /home/admin/mtr/script_for_cron.py -j default -m 20 -a 'python3 ~/workarea/git/Velours/python/prod/datou.py -j batch_current -C 2525467' -s traitement_3459 -M 0 -S 0 -U 100,80,95 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/home/admin/workarea/git/apy', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3473826 load datou : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : step 0 init_dummy_multi_datou is not linked in the step_by_step architecture ! WARNING : step 1294 init_dummy_multi_datou is not linked in the step_by_step architecture ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : (photo_id, hashtag_id, score_max) was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3459, datou_cur_ids : ['2525467'] with mtr_portfolio_ids : ['20038647'] and first list_photo_ids : [] new path : /proc/3473826/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, thcl, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 20 list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 20 ; length of list_pids : 20 ; length of list_args : 20 time to download the photos : 3.9608592987060547 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Wed Feb 5 11:32:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10332 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-05 11:32:31.248112: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-05 11:32:31.275298: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-05 11:32:31.277278: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f72c4000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-05 11:32:31.277308: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-05 11:32:31.281323: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-05 11:32:31.521975: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x4044e460 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-05 11:32:31.522032: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-05 11:32:31.523318: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-05 11:32:31.523757: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-05 11:32:31.526880: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-05 11:32:31.529926: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-05 11:32:31.530418: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-05 11:32:31.533974: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-05 11:32:31.535785: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-05 11:32:31.540364: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-05 11:32:31.541822: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-05 11:32:31.541899: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-05 11:32:31.542641: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-05 11:32:31.542657: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-05 11:32:31.542684: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-05 11:32:31.544037: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9570 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-05 11:32:31.825794: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-05 11:32:31.825913: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-05 11:32:31.825938: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-05 11:32:31.825960: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-05 11:32:31.825981: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-05 11:32:31.826001: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-05 11:32:31.826020: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-05 11:32:31.826041: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-05 11:32:31.827791: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-05 11:32:31.829317: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-05 11:32:31.829367: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-05 11:32:31.829387: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-05 11:32:31.829404: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-05 11:32:31.829421: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-05 11:32:31.829438: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-05 11:32:31.829456: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-05 11:32:31.829473: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-05 11:32:31.831064: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-05 11:32:31.831107: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-05 11:32:31.831118: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-05 11:32:31.831128: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-05 11:32:31.832641: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9570 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-05 11:32:39.524754: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-05 11:32:39.717082: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 20 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 145.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 12 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 12 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 144.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 11 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 14 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 11 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 149.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 147.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 146.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 148.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 12 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 8 Detection mask done ! Trying to reset tf kernel 3474446 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5043 tf kernel not reseted sub process len(results) : 20 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 20 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10332 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.00019788742065429688 nb_pixel_total : 4605 time to create 1 rle with old method : 0.005932807922363281 length of segment : 104 time for calcul the mask position with numpy : 0.008775949478149414 nb_pixel_total : 478664 time to create 1 rle with new method : 0.02188420295715332 length of segment : 986 time for calcul the mask position with numpy : 0.00010466575622558594 nb_pixel_total : 3817 time to create 1 rle with old method : 0.0047609806060791016 length of segment : 46 time for calcul the mask position with numpy : 0.0008177757263183594 nb_pixel_total : 34951 time to create 1 rle with old method : 0.039237022399902344 length of segment : 218 time for calcul the mask position with numpy : 0.0001220703125 nb_pixel_total : 1731 time to create 1 rle with old method : 0.0022034645080566406 length of segment : 83 time for calcul the mask position with numpy : 0.0003414154052734375 nb_pixel_total : 20831 time to create 1 rle with old method : 0.024192094802856445 length of segment : 94 time for calcul the mask position with numpy : 0.0008203983306884766 nb_pixel_total : 53468 time to create 1 rle with old method : 0.059393882751464844 length of segment : 216 time for calcul the mask position with numpy : 0.00016498565673828125 nb_pixel_total : 7557 time to create 1 rle with old method : 0.008709192276000977 length of segment : 94 time for calcul the mask position with numpy : 0.0005354881286621094 nb_pixel_total : 28011 time to create 1 rle with old method : 0.031646013259887695 length of segment : 241 time for calcul the mask position with numpy : 0.005560159683227539 nb_pixel_total : 421093 time to create 1 rle with new method : 0.013248920440673828 length of segment : 1285 time for calcul the mask position with numpy : 0.0001316070556640625 nb_pixel_total : 4965 time to create 1 rle with old method : 0.005873918533325195 length of segment : 69 time for calcul the mask position with numpy : 0.00015401840209960938 nb_pixel_total : 5174 time to create 1 rle with old method : 0.006017208099365234 length of segment : 126 time for calcul the mask position with numpy : 0.008064031600952148 nb_pixel_total : 523732 time to create 1 rle with new method : 0.017019271850585938 length of segment : 1308 time for calcul the mask position with numpy : 0.0007398128509521484 nb_pixel_total : 40825 time to create 1 rle with old method : 0.04559969902038574 length of segment : 315 time for calcul the mask position with numpy : 0.00046133995056152344 nb_pixel_total : 28299 time to create 1 rle with old method : 0.031084537506103516 length of segment : 129 time for calcul the mask position with numpy : 6.67572021484375e-05 nb_pixel_total : 1431 time to create 1 rle with old method : 0.0017077922821044922 length of segment : 54 time for calcul the mask position with numpy : 0.00039386749267578125 nb_pixel_total : 29011 time to create 1 rle with old method : 0.031755924224853516 length of segment : 126 time for calcul the mask position with numpy : 0.00029397010803222656 nb_pixel_total : 15810 time to create 1 rle with old method : 0.01715373992919922 length of segment : 236 time for calcul the mask position with numpy : 0.00018787384033203125 nb_pixel_total : 4149 time to create 1 rle with old method : 0.004873991012573242 length of segment : 71 time for calcul the mask position with numpy : 0.007679939270019531 nb_pixel_total : 411813 time to create 1 rle with new method : 0.026027441024780273 length of segment : 1350 time for calcul the mask position with numpy : 0.00021529197692871094 nb_pixel_total : 11085 time to create 1 rle with old method : 0.012147665023803711 length of segment : 115 time for calcul the mask position with numpy : 0.0014982223510742188 nb_pixel_total : 86069 time to create 1 rle with old method : 0.09451580047607422 length of segment : 699 time for calcul the mask position with numpy : 0.0002872943878173828 nb_pixel_total : 13099 time to create 1 rle with old method : 0.015163898468017578 length of segment : 194 time for calcul the mask position with numpy : 0.00013327598571777344 nb_pixel_total : 5958 time to create 1 rle with old method : 0.0072247982025146484 length of segment : 64 time for calcul the mask position with numpy : 0.0003714561462402344 nb_pixel_total : 17538 time to create 1 rle with old method : 0.01995849609375 length of segment : 152 time for calcul the mask position with numpy : 0.0008063316345214844 nb_pixel_total : 41202 time to create 1 rle with old method : 0.04546093940734863 length of segment : 356 time for calcul the mask position with numpy : 0.0006592273712158203 nb_pixel_total : 39426 time to create 1 rle with old method : 0.04365682601928711 length of segment : 354 time for calcul the mask position with numpy : 0.00014710426330566406 nb_pixel_total : 5926 time to create 1 rle with old method : 0.0070667266845703125 length of segment : 68 time for calcul the mask position with numpy : 0.0002269744873046875 nb_pixel_total : 11661 time to create 1 rle with old method : 0.013468742370605469 length of segment : 122 time for calcul the mask position with numpy : 0.0009157657623291016 nb_pixel_total : 14648 time to create 1 rle with old method : 0.018066883087158203 length of segment : 222 time for calcul the mask position with numpy : 0.002219676971435547 nb_pixel_total : 80932 time to create 1 rle with old method : 0.08912897109985352 length of segment : 568 time for calcul the mask position with numpy : 0.0007023811340332031 nb_pixel_total : 23586 time to create 1 rle with old method : 0.02647995948791504 length of segment : 186 time for calcul the mask position with numpy : 0.0005559921264648438 nb_pixel_total : 26671 time to create 1 rle with old method : 0.030410289764404297 length of segment : 251 time for calcul the mask position with numpy : 0.0008592605590820312 nb_pixel_total : 6710 time to create 1 rle with old method : 0.007977962493896484 length of segment : 92 time for calcul the mask position with numpy : 0.0004878044128417969 nb_pixel_total : 26565 time to create 1 rle with old method : 0.03065204620361328 length of segment : 117 time for calcul the mask position with numpy : 0.0010008811950683594 nb_pixel_total : 39557 time to create 1 rle with old method : 0.04505181312561035 length of segment : 385 time for calcul the mask position with numpy : 0.00791788101196289 nb_pixel_total : 127475 time to create 1 rle with old method : 0.14017224311828613 length of segment : 1019 time for calcul the mask position with numpy : 0.004265785217285156 nb_pixel_total : 115712 time to create 1 rle with old method : 0.12809085845947266 length of segment : 868 time for calcul the mask position with numpy : 0.0007929801940917969 nb_pixel_total : 52420 time to create 1 rle with old method : 0.06184649467468262 length of segment : 318 time for calcul the mask position with numpy : 0.0006253719329833984 nb_pixel_total : 23066 time to create 1 rle with old method : 0.026723861694335938 length of segment : 272 time for calcul the mask position with numpy : 0.011263608932495117 nb_pixel_total : 621156 time to create 1 rle with new method : 0.02579212188720703 length of segment : 2190 time for calcul the mask position with numpy : 0.00019407272338867188 nb_pixel_total : 3279 time to create 1 rle with old method : 0.00425410270690918 length of segment : 52 time for calcul the mask position with numpy : 5.984306335449219e-05 nb_pixel_total : 649 time to create 1 rle with old method : 0.00083160400390625 length of segment : 52 time for calcul the mask position with numpy : 0.0003871917724609375 nb_pixel_total : 11844 time to create 1 rle with old method : 0.013691902160644531 length of segment : 141 time for calcul the mask position with numpy : 0.011930465698242188 nb_pixel_total : 589254 time to create 1 rle with new method : 0.01855015754699707 length of segment : 963 time for calcul the mask position with numpy : 0.0011746883392333984 nb_pixel_total : 63481 time to create 1 rle with old method : 0.07056117057800293 length of segment : 339 time for calcul the mask position with numpy : 0.00010132789611816406 nb_pixel_total : 1037 time to create 1 rle with old method : 0.001346588134765625 length of segment : 47 time for calcul the mask position with numpy : 0.0012574195861816406 nb_pixel_total : 79906 time to create 1 rle with old method : 0.08858823776245117 length of segment : 391 time for calcul the mask position with numpy : 0.0002999305725097656 nb_pixel_total : 10539 time to create 1 rle with old method : 0.012709379196166992 length of segment : 119 time for calcul the mask position with numpy : 0.0002014636993408203 nb_pixel_total : 12876 time to create 1 rle with old method : 0.015496969223022461 length of segment : 180 time for calcul the mask position with numpy : 0.0003285408020019531 nb_pixel_total : 2248 time to create 1 rle with old method : 0.00283050537109375 length of segment : 141 time for calcul the mask position with numpy : 0.0004086494445800781 nb_pixel_total : 10123 time to create 1 rle with old method : 0.01199483871459961 length of segment : 151 time for calcul the mask position with numpy : 0.0001614093780517578 nb_pixel_total : 3596 time to create 1 rle with old method : 0.0044782161712646484 length of segment : 71 time for calcul the mask position with numpy : 0.002321004867553711 nb_pixel_total : 57246 time to create 1 rle with old method : 0.08210396766662598 length of segment : 541 time for calcul the mask position with numpy : 0.01310420036315918 nb_pixel_total : 707485 time to create 1 rle with new method : 0.02207326889038086 length of segment : 1360 time for calcul the mask position with numpy : 0.0009930133819580078 nb_pixel_total : 53056 time to create 1 rle with old method : 0.06180691719055176 length of segment : 502 time for calcul the mask position with numpy : 0.0007691383361816406 nb_pixel_total : 35003 time to create 1 rle with old method : 0.04079031944274902 length of segment : 261 time for calcul the mask position with numpy : 0.0011501312255859375 nb_pixel_total : 66759 time to create 1 rle with old method : 0.07757806777954102 length of segment : 234 time for calcul the mask position with numpy : 6.127357482910156e-05 nb_pixel_total : 1163 time to create 1 rle with old method : 0.0015213489532470703 length of segment : 45 time for calcul the mask position with numpy : 6.651878356933594e-05 nb_pixel_total : 1346 time to create 1 rle with old method : 0.0016131401062011719 length of segment : 74 time for calcul the mask position with numpy : 0.00035262107849121094 nb_pixel_total : 9588 time to create 1 rle with old method : 0.011961221694946289 length of segment : 172 time for calcul the mask position with numpy : 0.0004868507385253906 nb_pixel_total : 12715 time to create 1 rle with old method : 0.01477503776550293 length of segment : 156 time for calcul the mask position with numpy : 0.0008285045623779297 nb_pixel_total : 28641 time to create 1 rle with old method : 0.03306460380554199 length of segment : 259 time for calcul the mask position with numpy : 0.0005736351013183594 nb_pixel_total : 18396 time to create 1 rle with old method : 0.02108478546142578 length of segment : 214 time for calcul the mask position with numpy : 0.001001596450805664 nb_pixel_total : 32041 time to create 1 rle with old method : 0.037119150161743164 length of segment : 312 time for calcul the mask position with numpy : 0.0003523826599121094 nb_pixel_total : 5741 time to create 1 rle with old method : 0.007219076156616211 length of segment : 140 time for calcul the mask position with numpy : 0.0009374618530273438 nb_pixel_total : 27833 time to create 1 rle with old method : 0.032523393630981445 length of segment : 360 time for calcul the mask position with numpy : 0.007361650466918945 nb_pixel_total : 351988 time to create 1 rle with new method : 0.013820648193359375 length of segment : 993 time for calcul the mask position with numpy : 0.005213022232055664 nb_pixel_total : 105677 time to create 1 rle with old method : 0.12015366554260254 length of segment : 898 time for calcul the mask position with numpy : 0.0009253025054931641 nb_pixel_total : 17710 time to create 1 rle with old method : 0.0215151309967041 length of segment : 421 time for calcul the mask position with numpy : 0.00032401084899902344 nb_pixel_total : 7304 time to create 1 rle with old method : 0.009126424789428711 length of segment : 86 time for calcul the mask position with numpy : 0.0002605915069580078 nb_pixel_total : 3973 time to create 1 rle with old method : 0.004847526550292969 length of segment : 117 time for calcul the mask position with numpy : 0.01587986946105957 nb_pixel_total : 689943 time to create 1 rle with new method : 0.026077747344970703 length of segment : 1398 time for calcul the mask position with numpy : 0.0003008842468261719 nb_pixel_total : 8916 time to create 1 rle with old method : 0.01082468032836914 length of segment : 67 time for calcul the mask position with numpy : 0.0006263256072998047 nb_pixel_total : 36227 time to create 1 rle with old method : 0.04160451889038086 length of segment : 350 time for calcul the mask position with numpy : 0.00020122528076171875 nb_pixel_total : 3771 time to create 1 rle with old method : 0.004687070846557617 length of segment : 83 time for calcul the mask position with numpy : 0.0013432502746582031 nb_pixel_total : 53937 time to create 1 rle with old method : 0.06151413917541504 length of segment : 494 time for calcul the mask position with numpy : 0.0012590885162353516 nb_pixel_total : 69367 time to create 1 rle with old method : 0.07878327369689941 length of segment : 393 time for calcul the mask position with numpy : 0.00035881996154785156 nb_pixel_total : 11792 time to create 1 rle with old method : 0.013981819152832031 length of segment : 217 time for calcul the mask position with numpy : 8.940696716308594e-05 nb_pixel_total : 998 time to create 1 rle with old method : 0.001360177993774414 length of segment : 99 time for calcul the mask position with numpy : 0.0007436275482177734 nb_pixel_total : 32903 time to create 1 rle with old method : 0.03675365447998047 length of segment : 217 time for calcul the mask position with numpy : 0.01247859001159668 nb_pixel_total : 582966 time to create 1 rle with new method : 0.019156694412231445 length of segment : 1355 time for calcul the mask position with numpy : 0.0010085105895996094 nb_pixel_total : 64548 time to create 1 rle with old method : 0.07227039337158203 length of segment : 212 time for calcul the mask position with numpy : 0.0034248828887939453 nb_pixel_total : 184103 time to create 1 rle with new method : 0.007457256317138672 length of segment : 659 time for calcul the mask position with numpy : 0.00013303756713867188 nb_pixel_total : 4218 time to create 1 rle with old method : 0.005361795425415039 length of segment : 93 time for calcul the mask position with numpy : 0.0016713142395019531 nb_pixel_total : 87692 time to create 1 rle with old method : 0.09811186790466309 length of segment : 582 time for calcul the mask position with numpy : 0.0003001689910888672 nb_pixel_total : 17359 time to create 1 rle with old method : 0.0209810733795166 length of segment : 188 time for calcul the mask position with numpy : 0.0005130767822265625 nb_pixel_total : 32567 time to create 1 rle with old method : 0.037973880767822266 length of segment : 272 time for calcul the mask position with numpy : 0.0009744167327880859 nb_pixel_total : 71427 time to create 1 rle with old method : 0.08260774612426758 length of segment : 280 time for calcul the mask position with numpy : 0.002417325973510742 nb_pixel_total : 169942 time to create 1 rle with new method : 0.005773782730102539 length of segment : 571 time for calcul the mask position with numpy : 0.002655029296875 nb_pixel_total : 98549 time to create 1 rle with old method : 0.10760259628295898 length of segment : 589 time for calcul the mask position with numpy : 0.015774250030517578 nb_pixel_total : 577222 time to create 1 rle with new method : 0.027184486389160156 length of segment : 1152 time for calcul the mask position with numpy : 0.0009748935699462891 nb_pixel_total : 52831 time to create 1 rle with old method : 0.058431148529052734 length of segment : 280 time for calcul the mask position with numpy : 0.0009906291961669922 nb_pixel_total : 68368 time to create 1 rle with old method : 0.07517623901367188 length of segment : 454 time for calcul the mask position with numpy : 0.00044226646423339844 nb_pixel_total : 33714 time to create 1 rle with old method : 0.0379638671875 length of segment : 209 time for calcul the mask position with numpy : 0.004293680191040039 nb_pixel_total : 375168 time to create 1 rle with new method : 0.009578704833984375 length of segment : 1077 time for calcul the mask position with numpy : 0.0011546611785888672 nb_pixel_total : 67325 time to create 1 rle with old method : 0.07458043098449707 length of segment : 733 time for calcul the mask position with numpy : 0.0006327629089355469 nb_pixel_total : 37687 time to create 1 rle with old method : 0.04279947280883789 length of segment : 232 time for calcul the mask position with numpy : 0.0007965564727783203 nb_pixel_total : 64179 time to create 1 rle with old method : 0.07100558280944824 length of segment : 430 time for calcul the mask position with numpy : 0.012176275253295898 nb_pixel_total : 573217 time to create 1 rle with new method : 0.022431612014770508 length of segment : 1815 time for calcul the mask position with numpy : 0.00010132789611816406 nb_pixel_total : 3924 time to create 1 rle with old method : 0.0046923160552978516 length of segment : 50 time for calcul the mask position with numpy : 0.0007371902465820312 nb_pixel_total : 36853 time to create 1 rle with old method : 0.0412747859954834 length of segment : 494 time for calcul the mask position with numpy : 0.00021147727966308594 nb_pixel_total : 10438 time to create 1 rle with old method : 0.012583494186401367 length of segment : 171 time for calcul the mask position with numpy : 0.003610849380493164 nb_pixel_total : 319250 time to create 1 rle with new method : 0.00958704948425293 length of segment : 1294 time for calcul the mask position with numpy : 0.001474618911743164 nb_pixel_total : 93053 time to create 1 rle with old method : 0.10196781158447266 length of segment : 500 time for calcul the mask position with numpy : 0.0012669563293457031 nb_pixel_total : 71084 time to create 1 rle with old method : 0.08198332786560059 length of segment : 417 time for calcul the mask position with numpy : 0.0008192062377929688 nb_pixel_total : 32793 time to create 1 rle with old method : 0.036206960678100586 length of segment : 248 time for calcul the mask position with numpy : 0.0003325939178466797 nb_pixel_total : 14167 time to create 1 rle with old method : 0.016628503799438477 length of segment : 145 time for calcul the mask position with numpy : 0.012347936630249023 nb_pixel_total : 632270 time to create 1 rle with new method : 0.021535158157348633 length of segment : 1215 time for calcul the mask position with numpy : 0.0003490447998046875 nb_pixel_total : 19895 time to create 1 rle with old method : 0.022964000701904297 length of segment : 381 time for calcul the mask position with numpy : 0.0042459964752197266 nb_pixel_total : 376608 time to create 1 rle with new method : 0.009470224380493164 length of segment : 1041 time for calcul the mask position with numpy : 0.0017011165618896484 nb_pixel_total : 147105 time to create 1 rle with old method : 0.16801238059997559 length of segment : 393 time for calcul the mask position with numpy : 0.0009417533874511719 nb_pixel_total : 55823 time to create 1 rle with old method : 0.06343960762023926 length of segment : 423 time for calcul the mask position with numpy : 0.0005536079406738281 nb_pixel_total : 14881 time to create 1 rle with old method : 0.0184175968170166 length of segment : 172 time for calcul the mask position with numpy : 0.014661788940429688 nb_pixel_total : 704569 time to create 1 rle with new method : 0.02729654312133789 length of segment : 1539 time for calcul the mask position with numpy : 0.0007991790771484375 nb_pixel_total : 40902 time to create 1 rle with old method : 0.04853510856628418 length of segment : 348 time for calcul the mask position with numpy : 0.0037927627563476562 nb_pixel_total : 163634 time to create 1 rle with new method : 0.006175041198730469 length of segment : 389 time for calcul the mask position with numpy : 0.0012764930725097656 nb_pixel_total : 83564 time to create 1 rle with old method : 0.09396839141845703 length of segment : 410 time for calcul the mask position with numpy : 0.001798391342163086 nb_pixel_total : 117372 time to create 1 rle with old method : 0.1302628517150879 length of segment : 442 time for calcul the mask position with numpy : 0.0008358955383300781 nb_pixel_total : 48944 time to create 1 rle with old method : 0.055849313735961914 length of segment : 448 time for calcul the mask position with numpy : 0.0003485679626464844 nb_pixel_total : 25223 time to create 1 rle with old method : 0.028458356857299805 length of segment : 232 time for calcul the mask position with numpy : 0.009462833404541016 nb_pixel_total : 515630 time to create 1 rle with new method : 0.017740488052368164 length of segment : 1343 time for calcul the mask position with numpy : 0.001870870590209961 nb_pixel_total : 97345 time to create 1 rle with old method : 0.11123967170715332 length of segment : 628 time for calcul the mask position with numpy : 6.222724914550781e-05 nb_pixel_total : 902 time to create 1 rle with old method : 0.0012085437774658203 length of segment : 42 time for calcul the mask position with numpy : 0.0002148151397705078 nb_pixel_total : 4901 time to create 1 rle with old method : 0.005928993225097656 length of segment : 95 time for calcul the mask position with numpy : 9.298324584960938e-05 nb_pixel_total : 1324 time to create 1 rle with old method : 0.0019273757934570312 length of segment : 28 time for calcul the mask position with numpy : 0.0004177093505859375 nb_pixel_total : 12410 time to create 1 rle with old method : 0.014482975006103516 length of segment : 174 time for calcul the mask position with numpy : 0.0002541542053222656 nb_pixel_total : 6342 time to create 1 rle with old method : 0.007931232452392578 length of segment : 108 time for calcul the mask position with numpy : 0.0008249282836914062 nb_pixel_total : 24739 time to create 1 rle with old method : 0.02981400489807129 length of segment : 209 time for calcul the mask position with numpy : 0.00025916099548339844 nb_pixel_total : 10910 time to create 1 rle with old method : 0.013216972351074219 length of segment : 111 time for calcul the mask position with numpy : 0.0025107860565185547 nb_pixel_total : 80954 time to create 1 rle with old method : 0.0924522876739502 length of segment : 570 time for calcul the mask position with numpy : 0.0004913806915283203 nb_pixel_total : 24774 time to create 1 rle with old method : 0.029259920120239258 length of segment : 132 time for calcul the mask position with numpy : 0.0038759708404541016 nb_pixel_total : 182438 time to create 1 rle with new method : 0.007860898971557617 length of segment : 1089 time for calcul the mask position with numpy : 0.0001964569091796875 nb_pixel_total : 6821 time to create 1 rle with old method : 0.007941722869873047 length of segment : 89 time for calcul the mask position with numpy : 0.0008642673492431641 nb_pixel_total : 49848 time to create 1 rle with old method : 0.057814598083496094 length of segment : 139 time for calcul the mask position with numpy : 0.00106048583984375 nb_pixel_total : 43864 time to create 1 rle with old method : 0.050142765045166016 length of segment : 227 time for calcul the mask position with numpy : 0.0006606578826904297 nb_pixel_total : 27983 time to create 1 rle with old method : 0.03206038475036621 length of segment : 183 time for calcul the mask position with numpy : 0.00025177001953125 nb_pixel_total : 8857 time to create 1 rle with old method : 0.011272192001342773 length of segment : 82 time for calcul the mask position with numpy : 0.00218963623046875 nb_pixel_total : 104361 time to create 1 rle with old method : 0.1174323558807373 length of segment : 495 time for calcul the mask position with numpy : 0.0005033016204833984 nb_pixel_total : 28264 time to create 1 rle with old method : 0.032742977142333984 length of segment : 178 time for calcul the mask position with numpy : 0.0013580322265625 nb_pixel_total : 101039 time to create 1 rle with old method : 0.11217641830444336 length of segment : 481 time for calcul the mask position with numpy : 0.0010356903076171875 nb_pixel_total : 34922 time to create 1 rle with old method : 0.0401911735534668 length of segment : 295 time for calcul the mask position with numpy : 0.0006542205810546875 nb_pixel_total : 17201 time to create 1 rle with old method : 0.019880294799804688 length of segment : 160 time for calcul the mask position with numpy : 0.00046539306640625 nb_pixel_total : 11386 time to create 1 rle with old method : 0.013542413711547852 length of segment : 184 time for calcul the mask position with numpy : 0.00023746490478515625 nb_pixel_total : 2770 time to create 1 rle with old method : 0.003542184829711914 length of segment : 100 time for calcul the mask position with numpy : 0.0004572868347167969 nb_pixel_total : 18899 time to create 1 rle with old method : 0.022427082061767578 length of segment : 159 time for calcul the mask position with numpy : 0.00033855438232421875 nb_pixel_total : 7382 time to create 1 rle with old method : 0.009099960327148438 length of segment : 84 time for calcul the mask position with numpy : 0.0001709461212158203 nb_pixel_total : 3234 time to create 1 rle with old method : 0.004235267639160156 length of segment : 73 time for calcul the mask position with numpy : 0.017551898956298828 nb_pixel_total : 675123 time to create 1 rle with new method : 0.029599666595458984 length of segment : 1612 time for calcul the mask position with numpy : 7.104873657226562e-05 nb_pixel_total : 1520 time to create 1 rle with old method : 0.0019659996032714844 length of segment : 39 time for calcul the mask position with numpy : 0.003674745559692383 nb_pixel_total : 299834 time to create 1 rle with new method : 0.008472442626953125 length of segment : 1029 time for calcul the mask position with numpy : 0.0006825923919677734 nb_pixel_total : 26893 time to create 1 rle with old method : 0.0390164852142334 length of segment : 248 time for calcul the mask position with numpy : 0.00035881996154785156 nb_pixel_total : 13397 time to create 1 rle with old method : 0.016012191772460938 length of segment : 154 time for calcul the mask position with numpy : 0.0003330707550048828 nb_pixel_total : 8781 time to create 1 rle with old method : 0.010477066040039062 length of segment : 151 time for calcul the mask position with numpy : 0.0008857250213623047 nb_pixel_total : 47473 time to create 1 rle with old method : 0.05525064468383789 length of segment : 235 time for calcul the mask position with numpy : 0.004983425140380859 nb_pixel_total : 293360 time to create 1 rle with new method : 0.008602380752563477 length of segment : 1139 time for calcul the mask position with numpy : 0.00024271011352539062 nb_pixel_total : 8023 time to create 1 rle with old method : 0.009681940078735352 length of segment : 87 time for calcul the mask position with numpy : 0.0005698204040527344 nb_pixel_total : 26500 time to create 1 rle with old method : 0.030159473419189453 length of segment : 163 time for calcul the mask position with numpy : 0.0019402503967285156 nb_pixel_total : 122994 time to create 1 rle with old method : 0.13562703132629395 length of segment : 605 time for calcul the mask position with numpy : 0.007555961608886719 nb_pixel_total : 413664 time to create 1 rle with new method : 0.017478227615356445 length of segment : 1737 time spent for convertir_results : 14.311753511428833 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 160 chid ids of type : 3663 Number RLEs to save : 65465 save missing photos in datou_result : time spend for datou_step_exec : 37.60795736312866 time spend to save output : 3.8633973598480225 total time spend for step 1 : 41.471354722976685 step2:crop_condition Wed Feb 5 11:33:09 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 20 ! batch 1 Loaded 160 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 10 About to insert : list_path_to_insert length 10 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 34 About to insert : list_path_to_insert length 34 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 16 About to insert : list_path_to_insert length 16 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1332579049, 1332579041, 1332579037, 1332578979, 1332578949, 1332578909, 1332578907, 1332578904, 1332578882, 1332578875, 1332578871, 1332578867, 1332578866, 1332578863, 1332578856, 1332578830, 1332578813, 1332578798, 1332578634, 1332578632] Looping around the photos to save general results len do output : 76 /-3655809320Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809325Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809350Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809371Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809385Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809380Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809403Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809401Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809402Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809455Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809334Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809343Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809357Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809396Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809416Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809435Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809446Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809374Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809415Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809437Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809456Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809324Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809323Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809321Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809336Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809344Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809340Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809356Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809361Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809369Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809366Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809368Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809375Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809384Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809392Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809395Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809393Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809405Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809406Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809412Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809410Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809423Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809429Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809438Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809434Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809436Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809451Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809444Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809458Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809453Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809452Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809470Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809469Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809476Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809475Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809341Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809349Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809359Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809388Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809391Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809400Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809418Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809425Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809424Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809431Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809433Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809454Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809463Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809462Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809464Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809467Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809332Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809351Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809353Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809367Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655809411Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332579049', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332579041', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332579037', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578979', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578949', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578909', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578907', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578904', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578882', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578875', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578871', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578867', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578866', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578863', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578856', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578830', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578813', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578798', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578634', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578632', None, None, None, None, None, '2525467') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 248 time used for this insertion : 0.03881478309631348 save_final save missing photos in datou_result : time spend for datou_step_exec : 17.24264407157898 time spend to save output : 0.04081988334655762 total time spend for step 2 : 17.283463954925537 step3:thcl Wed Feb 5 11:33:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 2 we are using the classfication for multi_thcl [2456, 2868] time to import caffe and check if the image exist : 0.005663633346557617 time to convert the images to numpy array : 0.04377293586730957 time to import caffe and check if the image exist : 0.007866859436035156 time to convert the images to numpy array : 0.04114127159118652 time to import caffe and check if the image exist : 0.009324073791503906 time to convert the images to numpy array : 0.04403328895568848 time to import caffe and check if the image exist : 0.015818357467651367 time to convert the images to numpy array : 0.039548397064208984 time to import caffe and check if the image exist : 0.008333921432495117 time to convert the images to numpy array : 0.0633244514465332 time to import caffe and check if the image exist : 0.011849164962768555 time to convert the images to numpy array : 0.06112933158874512 time to import caffe and check if the image exist : 0.010515928268432617 time to convert the images to numpy array : 0.0666360855102539 time to import caffe and check if the image exist : 0.016172170639038086 time to convert the images to numpy array : 0.061907291412353516 time to import caffe and check if the image exist : 0.011031150817871094 time to convert the images to numpy array : 0.07496809959411621 time to import caffe and check if the image exist : 0.00813150405883789 time to convert the images to numpy array : 0.08469510078430176 total time to convert the images to numpy array : 0.331392765045166 list photo_ids error: [] list photo_ids correct : [-3655809336, -3655809344, -3655809340, -3655809356, -3655809361, -3655809369, -3655809366, -3655809368, -3655809351, -3655809353, -3655809367, -3655809411, -3655809431, -3655809433, -3655809454, -3655809463, -3655809462, -3655809464, -3655809467, -3655809332, -3655809349, -3655809359, -3655809388, -3655809391, -3655809400, -3655809418, -3655809425, -3655809424, -3655809402, -3655809455, -3655809334, -3655809343, -3655809357, -3655809396, -3655809416, -3655809435, -3655809375, -3655809384, -3655809392, -3655809395, -3655809393, -3655809405, -3655809406, -3655809412, -3655809446, -3655809374, -3655809415, -3655809437, -3655809456, -3655809324, -3655809323, -3655809321, -3655809458, -3655809453, -3655809452, -3655809470, -3655809469, -3655809476, -3655809475, -3655809341, -3655809410, -3655809423, -3655809429, -3655809438, -3655809434, -3655809436, -3655809451, -3655809444, -3655809320, -3655809325, -3655809350, -3655809371, -3655809385, -3655809380, -3655809403, -3655809401] number of photos to traite : 76 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10332 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/:/home/admin/workarea/git/apy/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10113 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 0.6011819839477539 time used to do the prediction : 0.1676769256591797 we don't save the descriptors for this thcl 2456 tagging for thcl : 2868 To do loadFromThcl(), then load ParamDescType : thcl2868 thcls : [{'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5288 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) To loadFromThcl() : net_5288 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10111 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_papier_nantes_300421 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_papier_nantes_300421 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_papier_nantes_300421 /data/models_weight/learn_papier_nantes_300421/caffemodel size_local : 44791983 size in s3 : 44791983 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:13 caffemodel already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/deploy.prototxt size_local : 17255 size in s3 : 17255 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:12 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:14 mean.npy already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/synset_words.txt size_local : 331 size in s3 : 331 create time local : 2021-08-09 05:56:00 create time in s3 : 2021-08-06 19:22:12 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/:/home/admin/workarea/git/apy/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_papier_nantes_300421/deploy.prototxt caffemodel_filename : /data/models_weight/learn_papier_nantes_300421/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10111 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 0.6177594661712646 time used to do the prediction : 0.1462235450744629 we don't save the descriptors for this thcl 2868 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 1.6927719116210938e-05 save missing photos in datou_result : time spend for datou_step_exec : 8.416717529296875 time spend to save output : 0.0005700588226318359 total time spend for step 3 : 8.417287588119507 step4:merge_mask_thcl_custom Wed Feb 5 11:33:35 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 160 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1332579049, 1332579041, 1332579037, 1332578979, 1332578949, 1332578909, 1332578907, 1332578904, 1332578882, 1332578875, 1332578871, 1332578867, 1332578866, 1332578863, 1332578856, 1332578830, 1332578813, 1332578798, 1332578634, 1332578632] Looping around the photos to save general results len do output : 20 /1332579049Didn't retrieve data .Didn't retrieve data . /1332579041Didn't retrieve data .Didn't retrieve data . /1332579037Didn't retrieve data .Didn't retrieve data . /1332578979Didn't retrieve data .Didn't retrieve data . /1332578949Didn't retrieve data .Didn't retrieve data . /1332578909Didn't retrieve data .Didn't retrieve data . /1332578907Didn't retrieve data .Didn't retrieve data . /1332578904Didn't retrieve data .Didn't retrieve data . /1332578882Didn't retrieve data .Didn't retrieve data . /1332578875Didn't retrieve data .Didn't retrieve data . /1332578871Didn't retrieve data .Didn't retrieve data . /1332578867Didn't retrieve data .Didn't retrieve data . /1332578866Didn't retrieve data .Didn't retrieve data . /1332578863Didn't retrieve data .Didn't retrieve data . /1332578856Didn't retrieve data .Didn't retrieve data . /1332578830Didn't retrieve data .Didn't retrieve data . /1332578813Didn't retrieve data .Didn't retrieve data . /1332578798Didn't retrieve data .Didn't retrieve data . /1332578634Didn't retrieve data .Didn't retrieve data . /1332578632Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332579049', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332579041', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332579037', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578979', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578949', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578909', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578907', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578904', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578882', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578875', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578871', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578867', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578866', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578863', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578856', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578830', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578813', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578798', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578634', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578632', None, None, None, None, None, '2525467') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 60 time used for this insertion : 0.013204097747802734 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.13049793243408203 time spend to save output : 0.014015674591064453 total time spend for step 4 : 0.14451360702514648 step5:rle_unique_nms_with_priority Wed Feb 5 11:33:35 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 7 nb_hashtags : 4 time to prepare the origin masks : 0.4741373062133789 time for calcul the mask position with numpy : 0.01896500587463379 nb_pixel_total : 1476861 time to create 1 rle with new method : 0.03845524787902832 time for calcul the mask position with numpy : 0.007358551025390625 nb_pixel_total : 53262 time to create 1 rle with old method : 0.056340932846069336 time for calcul the mask position with numpy : 0.006100177764892578 nb_pixel_total : 20740 time to create 1 rle with old method : 0.022481679916381836 time for calcul the mask position with numpy : 0.005806922912597656 nb_pixel_total : 1720 time to create 1 rle with old method : 0.0019729137420654297 time for calcul the mask position with numpy : 0.00573277473449707 nb_pixel_total : 3804 time to create 1 rle with old method : 0.004139423370361328 time for calcul the mask position with numpy : 0.005975008010864258 nb_pixel_total : 34731 time to create 1 rle with old method : 0.03971242904663086 time for calcul the mask position with numpy : 0.009332895278930664 nb_pixel_total : 477904 time to create 1 rle with new method : 0.036646366119384766 time for calcul the mask position with numpy : 0.006148099899291992 nb_pixel_total : 4578 time to create 1 rle with old method : 0.00534820556640625 create new chi : 0.27355003356933594 time to delete rle : 0.013468503952026367 batch 1 Loaded 8 chid ids of type : 3726 Number RLEs to save : 4592 TO DO : save crop sub photo not yet done ! save time : 0.2847106456756592 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.26569509506225586 time for calcul the mask position with numpy : 0.02284836769104004 nb_pixel_total : 1607212 time to create 1 rle with new method : 0.04749870300292969 time for calcul the mask position with numpy : 0.008885622024536133 nb_pixel_total : 430950 time to create 1 rle with new method : 0.028973817825317383 time for calcul the mask position with numpy : 0.007439374923706055 nb_pixel_total : 7537 time to create 1 rle with old method : 0.010387420654296875 time for calcul the mask position with numpy : 0.006383419036865234 nb_pixel_total : 27901 time to create 1 rle with old method : 0.03323173522949219 create new chi : 0.16852855682373047 time to delete rle : 0.00040078163146972656 batch 1 Loaded 4 chid ids of type : 3726 Number RLEs to save : 4030 TO DO : save crop sub photo not yet done ! save time : 0.2556154727935791 nb_obj : 7 nb_hashtags : 2 time to prepare the origin masks : 0.5465712547302246 time for calcul the mask position with numpy : 0.020728111267089844 nb_pixel_total : 1468812 time to create 1 rle with new method : 0.04267239570617676 time for calcul the mask position with numpy : 0.0066301822662353516 nb_pixel_total : 1517 time to create 1 rle with old method : 0.0018155574798583984 time for calcul the mask position with numpy : 0.006368398666381836 nb_pixel_total : 28197 time to create 1 rle with old method : 0.03195762634277344 time for calcul the mask position with numpy : 0.006591796875 nb_pixel_total : 1420 time to create 1 rle with old method : 0.0017347335815429688 time for calcul the mask position with numpy : 0.006753683090209961 nb_pixel_total : 40703 time to create 1 rle with old method : 0.05054903030395508 time for calcul the mask position with numpy : 0.010728836059570312 nb_pixel_total : 522874 time to create 1 rle with new method : 0.03319692611694336 time for calcul the mask position with numpy : 0.006104230880737305 nb_pixel_total : 5164 time to create 1 rle with old method : 0.006070852279663086 time for calcul the mask position with numpy : 0.0062868595123291016 nb_pixel_total : 4913 time to create 1 rle with old method : 0.005744218826293945 create new chi : 0.2471010684967041 time to delete rle : 0.0005412101745605469 batch 1 Loaded 8 chid ids of type : 3726 Number RLEs to save : 5185 TO DO : save crop sub photo not yet done ! save time : 0.30797815322875977 nb_obj : 9 nb_hashtags : 5 time to prepare the origin masks : 0.8734288215637207 time for calcul the mask position with numpy : 0.0174868106842041 nb_pixel_total : 1482881 time to create 1 rle with new method : 0.04262876510620117 time for calcul the mask position with numpy : 0.01006460189819336 nb_pixel_total : 4106 time to create 1 rle with old method : 0.007968664169311523 time for calcul the mask position with numpy : 0.006004810333251953 nb_pixel_total : 336 time to create 1 rle with old method : 0.0005245208740234375 time for calcul the mask position with numpy : 0.006215810775756836 nb_pixel_total : 41161 time to create 1 rle with old method : 0.0433652400970459 time for calcul the mask position with numpy : 0.006966590881347656 nb_pixel_total : 13056 time to create 1 rle with old method : 0.014461994171142578 time for calcul the mask position with numpy : 0.007490634918212891 nb_pixel_total : 85839 time to create 1 rle with old method : 0.08869242668151855 time for calcul the mask position with numpy : 0.006224632263183594 nb_pixel_total : 11074 time to create 1 rle with old method : 0.011954784393310547 time for calcul the mask position with numpy : 0.008663177490234375 nb_pixel_total : 412849 time to create 1 rle with new method : 0.039005279541015625 time for calcul the mask position with numpy : 0.007159709930419922 nb_pixel_total : 16376 time to create 1 rle with old method : 0.01753973960876465 time for calcul the mask position with numpy : 0.006605863571166992 nb_pixel_total : 5922 time to create 1 rle with old method : 0.006479978561401367 create new chi : 0.36031365394592285 time to delete rle : 0.0006906986236572266 batch 1 Loaded 10 chid ids of type : 3726 Number RLEs to save : 7002 TO DO : save crop sub photo not yet done ! save time : 0.4265410900115967 nb_obj : 10 nb_hashtags : 5 time to prepare the origin masks : 1.1098575592041016 time for calcul the mask position with numpy : 0.01972675323486328 nb_pixel_total : 1698509 time to create 1 rle with new method : 0.04547286033630371 time for calcul the mask position with numpy : 0.006346464157104492 nb_pixel_total : 9324 time to create 1 rle with old method : 0.010654926300048828 time for calcul the mask position with numpy : 0.0062978267669677734 nb_pixel_total : 23048 time to create 1 rle with old method : 0.025110244750976562 time for calcul the mask position with numpy : 0.006526470184326172 nb_pixel_total : 39465 time to create 1 rle with old method : 0.04209709167480469 time for calcul the mask position with numpy : 0.0062961578369140625 nb_pixel_total : 23570 time to create 1 rle with old method : 0.025352954864501953 time for calcul the mask position with numpy : 0.007213592529296875 nb_pixel_total : 127426 time to create 1 rle with old method : 0.136674165725708 time for calcul the mask position with numpy : 0.006353139877319336 nb_pixel_total : 26473 time to create 1 rle with old method : 0.03235197067260742 time for calcul the mask position with numpy : 0.006930112838745117 nb_pixel_total : 6675 time to create 1 rle with old method : 0.007166385650634766 time for calcul the mask position with numpy : 0.006609439849853516 nb_pixel_total : 80813 time to create 1 rle with old method : 0.11178803443908691 time for calcul the mask position with numpy : 0.006440639495849609 nb_pixel_total : 23491 time to create 1 rle with old method : 0.02534008026123047 time for calcul the mask position with numpy : 0.006252288818359375 nb_pixel_total : 14806 time to create 1 rle with old method : 0.01559591293334961 create new chi : 0.5686805248260498 time to delete rle : 0.0007302761077880859 batch 1 Loaded 11 chid ids of type : 3726 Number RLEs to save : 7324 TO DO : save crop sub photo not yet done ! save time : 1.1483089923858643 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.428912878036499 time for calcul the mask position with numpy : 0.017844200134277344 nb_pixel_total : 1449896 time to create 1 rle with new method : 0.034856319427490234 time for calcul the mask position with numpy : 0.0072174072265625 nb_pixel_total : 190 time to create 1 rle with old method : 0.0026252269744873047 time for calcul the mask position with numpy : 0.0069119930267333984 nb_pixel_total : 3248 time to create 1 rle with old method : 0.005677700042724609 time for calcul the mask position with numpy : 0.012714147567749023 nb_pixel_total : 620266 time to create 1 rle with new method : 0.042558908462524414 create new chi : 0.13130974769592285 time to delete rle : 0.00045108795166015625 batch 1 Loaded 4 chid ids of type : 3726 Number RLEs to save : 5346 TO DO : save crop sub photo not yet done ! save time : 0.3400411605834961 nb_obj : 9 nb_hashtags : 6 time to prepare the origin masks : 0.7001650333404541 time for calcul the mask position with numpy : 0.020116567611694336 nb_pixel_total : 1379489 time to create 1 rle with new method : 0.04231715202331543 time for calcul the mask position with numpy : 0.0063724517822265625 nb_pixel_total : 2314 time to create 1 rle with old method : 0.0029196739196777344 time for calcul the mask position with numpy : 0.006242275238037109 nb_pixel_total : 1032 time to create 1 rle with old method : 0.0012125968933105469 time for calcul the mask position with numpy : 0.006222248077392578 nb_pixel_total : 3321 time to create 1 rle with old method : 0.003582477569580078 time for calcul the mask position with numpy : 0.0067632198333740234 nb_pixel_total : 63416 time to create 1 rle with old method : 0.06893110275268555 time for calcul the mask position with numpy : 0.006096601486206055 nb_pixel_total : 10068 time to create 1 rle with old method : 0.011306047439575195 time for calcul the mask position with numpy : 0.009768486022949219 nb_pixel_total : 589402 time to create 1 rle with new method : 0.0336613655090332 time for calcul the mask position with numpy : 0.0060977935791015625 nb_pixel_total : 11828 time to create 1 rle with old method : 0.013413190841674805 time for calcul the mask position with numpy : 0.006212472915649414 nb_pixel_total : 2236 time to create 1 rle with old method : 0.002519369125366211 time for calcul the mask position with numpy : 0.006078481674194336 nb_pixel_total : 10494 time to create 1 rle with old method : 0.011723041534423828 create new chi : 0.27480101585388184 time to delete rle : 0.0008547306060791016 batch 1 Loaded 10 chid ids of type : 3726 Number RLEs to save : 5225 TO DO : save crop sub photo not yet done ! save time : 0.33309459686279297 nb_obj : 7 nb_hashtags : 3 time to prepare the origin masks : 0.6720380783081055 time for calcul the mask position with numpy : 0.017432689666748047 nb_pixel_total : 1182310 time to create 1 rle with new method : 0.03359222412109375 time for calcul the mask position with numpy : 0.007191658020019531 nb_pixel_total : 33418 time to create 1 rle with old method : 0.035919189453125 time for calcul the mask position with numpy : 0.006055355072021484 nb_pixel_total : 1339 time to create 1 rle with old method : 0.0014564990997314453 time for calcul the mask position with numpy : 0.006179332733154297 nb_pixel_total : 8470 time to create 1 rle with old method : 0.009474992752075195 time for calcul the mask position with numpy : 0.006558656692504883 nb_pixel_total : 53041 time to create 1 rle with old method : 0.05996060371398926 time for calcul the mask position with numpy : 0.010603904724121094 nb_pixel_total : 66649 time to create 1 rle with old method : 0.06895136833190918 time for calcul the mask position with numpy : 0.011602640151977539 nb_pixel_total : 724779 time to create 1 rle with new method : 0.029474496841430664 time for calcul the mask position with numpy : 0.006905555725097656 nb_pixel_total : 3594 time to create 1 rle with old method : 0.003844022750854492 create new chi : 0.3170480728149414 time to delete rle : 0.0005428791046142578 batch 1 Loaded 8 chid ids of type : 3726 Number RLEs to save : 6125 TO DO : save crop sub photo not yet done ! save time : 0.38863372802734375 nb_obj : 8 nb_hashtags : 2 time to prepare the origin masks : 0.6046173572540283 time for calcul the mask position with numpy : 0.020125865936279297 nb_pixel_total : 1587086 time to create 1 rle with new method : 0.04223775863647461 time for calcul the mask position with numpy : 0.006430149078369141 nb_pixel_total : 27803 time to create 1 rle with old method : 0.033036231994628906 time for calcul the mask position with numpy : 0.00885915756225586 nb_pixel_total : 352035 time to create 1 rle with new method : 0.035111427307128906 time for calcul the mask position with numpy : 0.006348848342895508 nb_pixel_total : 28571 time to create 1 rle with old method : 0.030460357666015625 time for calcul the mask position with numpy : 0.006077766418457031 nb_pixel_total : 5720 time to create 1 rle with old method : 0.006304502487182617 time for calcul the mask position with numpy : 0.006192207336425781 nb_pixel_total : 31973 time to create 1 rle with old method : 0.03516578674316406 time for calcul the mask position with numpy : 0.0066297054290771484 nb_pixel_total : 18268 time to create 1 rle with old method : 0.022377729415893555 time for calcul the mask position with numpy : 0.006441354751586914 nb_pixel_total : 12600 time to create 1 rle with old method : 0.014124870300292969 time for calcul the mask position with numpy : 0.006775379180908203 nb_pixel_total : 9544 time to create 1 rle with old method : 0.011148691177368164 create new chi : 0.3065979480743408 time to delete rle : 0.0005862712860107422 batch 1 Loaded 9 chid ids of type : 3726 Number RLEs to save : 6337 TO DO : save crop sub photo not yet done ! save time : 0.36460447311401367 nb_obj : 11 nb_hashtags : 6 time to prepare the origin masks : 0.9504959583282471 time for calcul the mask position with numpy : 0.019533157348632812 nb_pixel_total : 1220765 time to create 1 rle with new method : 0.042028188705444336 time for calcul the mask position with numpy : 0.006501197814941406 nb_pixel_total : 6597 time to create 1 rle with old method : 0.007527828216552734 time for calcul the mask position with numpy : 0.006844997406005859 nb_pixel_total : 18497 time to create 1 rle with old method : 0.02108478546142578 time for calcul the mask position with numpy : 0.0066525936126708984 nb_pixel_total : 36111 time to create 1 rle with old method : 0.03986406326293945 time for calcul the mask position with numpy : 0.006693840026855469 nb_pixel_total : 18 time to create 1 rle with old method : 4.482269287109375e-05 time for calcul the mask position with numpy : 0.0065920352935791016 nb_pixel_total : 4672 time to create 1 rle with old method : 0.0056002140045166016 time for calcul the mask position with numpy : 0.006510734558105469 nb_pixel_total : 53650 time to create 1 rle with old method : 0.0569307804107666 time for calcul the mask position with numpy : 0.006000041961669922 nb_pixel_total : 8795 time to create 1 rle with old method : 0.00961756706237793 time for calcul the mask position with numpy : 0.010799169540405273 nb_pixel_total : 699171 time to create 1 rle with new method : 0.036028385162353516 time for calcul the mask position with numpy : 0.0060214996337890625 nb_pixel_total : 3759 time to create 1 rle with old method : 0.0040090084075927734 time for calcul the mask position with numpy : 0.006054878234863281 nb_pixel_total : 3963 time to create 1 rle with old method : 0.004223823547363281 time for calcul the mask position with numpy : 0.005998373031616211 nb_pixel_total : 17602 time to create 1 rle with old method : 0.018978118896484375 create new chi : 0.343721866607666 time to delete rle : 0.0007021427154541016 batch 1 Loaded 12 chid ids of type : 3726 Number RLEs to save : 7238 TO DO : save crop sub photo not yet done ! save time : 0.4259781837463379 nb_obj : 9 nb_hashtags : 4 time to prepare the origin masks : 1.1984429359436035 time for calcul the mask position with numpy : 0.02224588394165039 nb_pixel_total : 1090653 time to create 1 rle with new method : 0.0483705997467041 time for calcul the mask position with numpy : 0.006837606430053711 nb_pixel_total : 20366 time to create 1 rle with old method : 0.021121978759765625 time for calcul the mask position with numpy : 0.006608486175537109 nb_pixel_total : 4525 time to create 1 rle with old method : 0.0048923492431640625 time for calcul the mask position with numpy : 0.007227182388305664 nb_pixel_total : 169678 time to create 1 rle with new method : 0.03382277488708496 time for calcul the mask position with numpy : 0.0067291259765625 nb_pixel_total : 32445 time to create 1 rle with old method : 0.03387808799743652 time for calcul the mask position with numpy : 0.006608247756958008 nb_pixel_total : 495 time to create 1 rle with old method : 0.0007200241088867188 time for calcul the mask position with numpy : 0.007194995880126953 nb_pixel_total : 32647 time to create 1 rle with old method : 0.0419306755065918 time for calcul the mask position with numpy : 0.006396770477294922 nb_pixel_total : 87257 time to create 1 rle with old method : 0.09158682823181152 time for calcul the mask position with numpy : 0.01135563850402832 nb_pixel_total : 602686 time to create 1 rle with new method : 0.02790665626525879 time for calcul the mask position with numpy : 0.006734609603881836 nb_pixel_total : 32848 time to create 1 rle with old method : 0.03494095802307129 create new chi : 0.43148303031921387 time to delete rle : 0.0010237693786621094 batch 1 Loaded 10 chid ids of type : 3726 Number RLEs to save : 7421 TO DO : save crop sub photo not yet done ! save time : 0.42885828018188477 nb_obj : 9 nb_hashtags : 4 time to prepare the origin masks : 1.0036921501159668 time for calcul the mask position with numpy : 0.013316631317138672 nb_pixel_total : 1059591 time to create 1 rle with new method : 0.038721561431884766 time for calcul the mask position with numpy : 0.0060274600982666016 nb_pixel_total : 342 time to create 1 rle with old method : 0.0005478858947753906 time for calcul the mask position with numpy : 0.006642341613769531 nb_pixel_total : 1843 time to create 1 rle with old method : 0.0024247169494628906 time for calcul the mask position with numpy : 0.008095741271972656 nb_pixel_total : 36451 time to create 1 rle with old method : 0.03949093818664551 time for calcul the mask position with numpy : 0.007901191711425781 nb_pixel_total : 165557 time to create 1 rle with new method : 0.030588150024414062 time for calcul the mask position with numpy : 0.006178855895996094 nb_pixel_total : 211 time to create 1 rle with old method : 0.00043654441833496094 time for calcul the mask position with numpy : 0.008131980895996094 nb_pixel_total : 64920 time to create 1 rle with old method : 0.07200455665588379 time for calcul the mask position with numpy : 0.006685972213745117 nb_pixel_total : 52775 time to create 1 rle with old method : 0.06083035469055176 time for calcul the mask position with numpy : 0.015410423278808594 nb_pixel_total : 588449 time to create 1 rle with new method : 0.032919883728027344 time for calcul the mask position with numpy : 0.00748753547668457 nb_pixel_total : 103461 time to create 1 rle with old method : 0.10608983039855957 create new chi : 0.47368788719177246 time to delete rle : 0.0006711483001708984 batch 1 Loaded 10 chid ids of type : 3726 Number RLEs to save : 7266 TO DO : save crop sub photo not yet done ! save time : 0.4392251968383789 nb_obj : 7 nb_hashtags : 2 time to prepare the origin masks : 0.7270975112915039 time for calcul the mask position with numpy : 0.019368410110473633 nb_pixel_total : 1228731 time to create 1 rle with new method : 0.039388179779052734 time for calcul the mask position with numpy : 0.0063931941986083984 nb_pixel_total : 51337 time to create 1 rle with old method : 0.054450035095214844 time for calcul the mask position with numpy : 0.00643610954284668 nb_pixel_total : 3914 time to create 1 rle with old method : 0.004312992095947266 time for calcul the mask position with numpy : 0.0062105655670166016 nb_pixel_total : 36892 time to create 1 rle with old method : 0.03990912437438965 time for calcul the mask position with numpy : 0.006250858306884766 nb_pixel_total : 60009 time to create 1 rle with old method : 0.0644536018371582 time for calcul the mask position with numpy : 0.0059528350830078125 nb_pixel_total : 10394 time to create 1 rle with old method : 0.01130819320678711 time for calcul the mask position with numpy : 0.006003141403198242 nb_pixel_total : 36698 time to create 1 rle with old method : 0.03795313835144043 time for calcul the mask position with numpy : 0.011389017105102539 nb_pixel_total : 645625 time to create 1 rle with new method : 0.030547618865966797 create new chi : 0.3538174629211426 time to delete rle : 0.0005888938903808594 batch 1 Loaded 8 chid ids of type : 3726 Number RLEs to save : 7204 TO DO : save crop sub photo not yet done ! save time : 0.5347747802734375 nb_obj : 7 nb_hashtags : 3 time to prepare the origin masks : 0.681816816329956 time for calcul the mask position with numpy : 0.012842178344726562 nb_pixel_total : 1033675 time to create 1 rle with new method : 0.04277515411376953 time for calcul the mask position with numpy : 0.006424427032470703 nb_pixel_total : 53669 time to create 1 rle with old method : 0.05674314498901367 time for calcul the mask position with numpy : 0.0068318843841552734 nb_pixel_total : 128446 time to create 1 rle with old method : 0.1396007537841797 time for calcul the mask position with numpy : 0.006116628646850586 nb_pixel_total : 14034 time to create 1 rle with old method : 0.01584029197692871 time for calcul the mask position with numpy : 0.006122112274169922 nb_pixel_total : 32691 time to create 1 rle with old method : 0.03472161293029785 time for calcul the mask position with numpy : 0.006142616271972656 nb_pixel_total : 19829 time to create 1 rle with old method : 0.021520137786865234 time for calcul the mask position with numpy : 0.00691676139831543 nb_pixel_total : 146661 time to create 1 rle with old method : 0.1604478359222412 time for calcul the mask position with numpy : 0.010522603988647461 nb_pixel_total : 644595 time to create 1 rle with new method : 0.04074382781982422 create new chi : 0.5834286212921143 time to delete rle : 0.0007739067077636719 batch 1 Loaded 8 chid ids of type : 3726 Number RLEs to save : 7004 TO DO : save crop sub photo not yet done ! save time : 0.41591382026672363 nb_obj : 8 nb_hashtags : 7 time to prepare the origin masks : 0.895402193069458 time for calcul the mask position with numpy : 0.011950016021728516 nb_pixel_total : 962005 time to create 1 rle with new method : 0.04356026649475098 time for calcul the mask position with numpy : 0.007133960723876953 nb_pixel_total : 161939 time to create 1 rle with new method : 0.03572893142700195 time for calcul the mask position with numpy : 0.0059545040130615234 nb_pixel_total : 457 time to create 1 rle with old method : 0.0007383823394775391 time for calcul the mask position with numpy : 0.006635189056396484 nb_pixel_total : 26152 time to create 1 rle with old method : 0.028003454208374023 time for calcul the mask position with numpy : 0.00613856315612793 nb_pixel_total : 48836 time to create 1 rle with old method : 0.051664113998413086 time for calcul the mask position with numpy : 0.005957603454589844 nb_pixel_total : 14840 time to create 1 rle with old method : 0.015527009963989258 time for calcul the mask position with numpy : 0.005979061126708984 nb_pixel_total : 24848 time to create 1 rle with old method : 0.026247501373291016 time for calcul the mask position with numpy : 0.012419462203979492 nb_pixel_total : 717591 time to create 1 rle with new method : 0.027344703674316406 time for calcul the mask position with numpy : 0.006525278091430664 nb_pixel_total : 116932 time to create 1 rle with old method : 0.12321805953979492 create new chi : 0.4261176586151123 time to delete rle : 0.00072479248046875 batch 1 Loaded 9 chid ids of type : 3726 Number RLEs to save : 7739 TO DO : save crop sub photo not yet done ! save time : 0.42563700675964355 nb_obj : 4 nb_hashtags : 1 time to prepare the origin masks : 0.30230140686035156 time for calcul the mask position with numpy : 0.016060829162597656 nb_pixel_total : 1467755 time to create 1 rle with new method : 0.029277563095092773 time for calcul the mask position with numpy : 0.006234169006347656 nb_pixel_total : 4895 time to create 1 rle with old method : 0.0054972171783447266 time for calcul the mask position with numpy : 0.0057833194732666016 nb_pixel_total : 901 time to create 1 rle with old method : 0.0011038780212402344 time for calcul the mask position with numpy : 0.005856752395629883 nb_pixel_total : 4725 time to create 1 rle with old method : 0.005456447601318359 time for calcul the mask position with numpy : 0.009973764419555664 nb_pixel_total : 595324 time to create 1 rle with new method : 0.027567386627197266 create new chi : 0.11584806442260742 time to delete rle : 0.0003762245178222656 batch 1 Loaded 5 chid ids of type : 3726 Number RLEs to save : 3466 TO DO : save crop sub photo not yet done ! save time : 0.20557522773742676 nb_obj : 7 nb_hashtags : 3 time to prepare the origin masks : 0.6152825355529785 time for calcul the mask position with numpy : 0.022850751876831055 nb_pixel_total : 1814626 time to create 1 rle with new method : 0.043007850646972656 time for calcul the mask position with numpy : 0.006475210189819336 nb_pixel_total : 24642 time to create 1 rle with old method : 0.02604508399963379 time for calcul the mask position with numpy : 0.007198810577392578 nb_pixel_total : 182795 time to create 1 rle with new method : 0.03546643257141113 time for calcul the mask position with numpy : 0.00595402717590332 nb_pixel_total : 24742 time to create 1 rle with old method : 0.02642226219177246 time for calcul the mask position with numpy : 0.00651097297668457 nb_pixel_total : 6322 time to create 1 rle with old method : 0.007128715515136719 time for calcul the mask position with numpy : 0.005761861801147461 nb_pixel_total : 6797 time to create 1 rle with old method : 0.008318901062011719 time for calcul the mask position with numpy : 0.00583338737487793 nb_pixel_total : 1318 time to create 1 rle with old method : 0.0015897750854492188 time for calcul the mask position with numpy : 0.0060579776763916016 nb_pixel_total : 12358 time to create 1 rle with old method : 0.01307988166809082 create new chi : 0.23321747779846191 time to delete rle : 0.00047898292541503906 batch 1 Loaded 8 chid ids of type : 3726 Number RLEs to save : 4716 TO DO : save crop sub photo not yet done ! save time : 0.30138087272644043 nb_obj : 6 nb_hashtags : 5 time to prepare the origin masks : 0.3635373115539551 time for calcul the mask position with numpy : 0.028513431549072266 nb_pixel_total : 1838233 time to create 1 rle with new method : 0.06211113929748535 time for calcul the mask position with numpy : 0.008594036102294922 nb_pixel_total : 43706 time to create 1 rle with old method : 0.06735849380493164 time for calcul the mask position with numpy : 0.006677865982055664 nb_pixel_total : 1375 time to create 1 rle with old method : 0.002042531967163086 time for calcul the mask position with numpy : 0.006608009338378906 nb_pixel_total : 49587 time to create 1 rle with old method : 0.05584359169006348 time for calcul the mask position with numpy : 0.006094932556152344 nb_pixel_total : 8793 time to create 1 rle with old method : 0.010097503662109375 time for calcul the mask position with numpy : 0.006548881530761719 nb_pixel_total : 104048 time to create 1 rle with old method : 0.10998821258544922 time for calcul the mask position with numpy : 0.006186723709106445 nb_pixel_total : 27858 time to create 1 rle with old method : 0.0301053524017334 create new chi : 0.4139668941497803 time to delete rle : 0.0003674030303955078 batch 1 Loaded 7 chid ids of type : 3726 Number RLEs to save : 3531 TO DO : save crop sub photo not yet done ! save time : 0.3096134662628174 nb_obj : 12 nb_hashtags : 3 time to prepare the origin masks : 1.065305471420288 time for calcul the mask position with numpy : 0.021764755249023438 nb_pixel_total : 1223755 time to create 1 rle with new method : 0.04364752769470215 time for calcul the mask position with numpy : 0.007185935974121094 nb_pixel_total : 1515 time to create 1 rle with old method : 0.001773834228515625 time for calcul the mask position with numpy : 0.005864858627319336 nb_pixel_total : 2712 time to create 1 rle with old method : 0.003083944320678711 time for calcul the mask position with numpy : 0.005818843841552734 nb_pixel_total : 560 time to create 1 rle with old method : 0.0007445812225341797 time for calcul the mask position with numpy : 0.006093740463256836 nb_pixel_total : 26853 time to create 1 rle with old method : 0.02918839454650879 time for calcul the mask position with numpy : 0.006285905838012695 nb_pixel_total : 11405 time to create 1 rle with old method : 0.012172937393188477 time for calcul the mask position with numpy : 0.0060749053955078125 nb_pixel_total : 37865 time to create 1 rle with old method : 0.041758060455322266 time for calcul the mask position with numpy : 0.01083230972290039 nb_pixel_total : 694708 time to create 1 rle with new method : 0.033773183822631836 time for calcul the mask position with numpy : 0.007004737854003906 nb_pixel_total : 3184 time to create 1 rle with old method : 0.0035512447357177734 time for calcul the mask position with numpy : 0.0057048797607421875 nb_pixel_total : 11319 time to create 1 rle with old method : 0.012117385864257812 time for calcul the mask position with numpy : 0.005803346633911133 nb_pixel_total : 18745 time to create 1 rle with old method : 0.018926382064819336 time for calcul the mask position with numpy : 0.005815267562866211 nb_pixel_total : 6243 time to create 1 rle with old method : 0.006800174713134766 time for calcul the mask position with numpy : 0.005738735198974609 nb_pixel_total : 34736 time to create 1 rle with old method : 0.03767824172973633 create new chi : 0.3495767116546631 time to delete rle : 0.0007691383361816406 batch 1 Loaded 13 chid ids of type : 3726 Number RLEs to save : 6883 TO DO : save crop sub photo not yet done ! save time : 0.49018120765686035 nb_obj : 7 nb_hashtags : 3 time to prepare the origin masks : 0.6658420562744141 time for calcul the mask position with numpy : 0.015190362930297852 nb_pixel_total : 1421715 time to create 1 rle with new method : 0.02935504913330078 time for calcul the mask position with numpy : 0.0057277679443359375 nb_pixel_total : 14052 time to create 1 rle with old method : 0.01527261734008789 time for calcul the mask position with numpy : 0.005733013153076172 nb_pixel_total : 26367 time to create 1 rle with old method : 0.028402090072631836 time for calcul the mask position with numpy : 0.007175922393798828 nb_pixel_total : 254565 time to create 1 rle with new method : 0.028311491012573242 time for calcul the mask position with numpy : 0.006871461868286133 nb_pixel_total : 8007 time to create 1 rle with old method : 0.009263753890991211 time for calcul the mask position with numpy : 0.007302522659301758 nb_pixel_total : 292704 time to create 1 rle with new method : 0.027715682983398438 time for calcul the mask position with numpy : 0.006207942962646484 nb_pixel_total : 47460 time to create 1 rle with old method : 0.050429344177246094 time for calcul the mask position with numpy : 0.0059168338775634766 nb_pixel_total : 8730 time to create 1 rle with old method : 0.009942293167114258 create new chi : 0.2598409652709961 time to delete rle : 0.0006175041198730469 batch 1 Loaded 8 chid ids of type : 3726 Number RLEs to save : 6567 TO DO : save crop sub photo not yet done ! save time : 0.5101556777954102 map_output_result : {1332579049: (0.0, 'Should be the crop_list due to order', 0.0), 1332579041: (0.0, 'Should be the crop_list due to order', 0.0), 1332579037: (0.0, 'Should be the crop_list due to order', 0.0), 1332578979: (0.0, 'Should be the crop_list due to order', 0.0), 1332578949: (0.0, 'Should be the crop_list due to order', 0.0), 1332578909: (0.0, 'Should be the crop_list due to order', 0.0), 1332578907: (0.0, 'Should be the crop_list due to order', 0.0), 1332578904: (0.0, 'Should be the crop_list due to order', 0.0), 1332578882: (0.0, 'Should be the crop_list due to order', 0.0), 1332578875: (0.0, 'Should be the crop_list due to order', 0.0), 1332578871: (0.0, 'Should be the crop_list due to order', 0.0), 1332578867: (0.0, 'Should be the crop_list due to order', 0.0), 1332578866: (0.0, 'Should be the crop_list due to order', 0.0), 1332578863: (0.0, 'Should be the crop_list due to order', 0.0), 1332578856: (0.0, 'Should be the crop_list due to order', 0.0), 1332578830: (0.0, 'Should be the crop_list due to order', 0.0), 1332578813: (0.0, 'Should be the crop_list due to order', 0.0), 1332578798: (0.0, 'Should be the crop_list due to order', 0.0), 1332578634: (0.0, 'Should be the crop_list due to order', 0.0), 1332578632: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1332579049, 1332579041, 1332579037, 1332578979, 1332578949, 1332578909, 1332578907, 1332578904, 1332578882, 1332578875, 1332578871, 1332578867, 1332578866, 1332578863, 1332578856, 1332578830, 1332578813, 1332578798, 1332578634, 1332578632] Looping around the photos to save general results len do output : 20 /1332579049.Didn't retrieve data . /1332579041.Didn't retrieve data . /1332579037.Didn't retrieve data . /1332578979.Didn't retrieve data . /1332578949.Didn't retrieve data . /1332578909.Didn't retrieve data . /1332578907.Didn't retrieve data . /1332578904.Didn't retrieve data . /1332578882.Didn't retrieve data . /1332578875.Didn't retrieve data . /1332578871.Didn't retrieve data . /1332578867.Didn't retrieve data . /1332578866.Didn't retrieve data . /1332578863.Didn't retrieve data . /1332578856.Didn't retrieve data . /1332578830.Didn't retrieve data . /1332578813.Didn't retrieve data . /1332578798.Didn't retrieve data . /1332578634.Didn't retrieve data . /1332578632.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332579049', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332579041', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332579037', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578979', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578949', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578909', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578907', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578904', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578882', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578875', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578871', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578867', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578866', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578863', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578856', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578830', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578813', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578798', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578634', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578632', None, None, None, None, None, '2525467') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 60 time used for this insertion : 0.013355255126953125 save_final save missing photos in datou_result : time spend for datou_step_exec : 29.875722646713257 time spend to save output : 0.013950824737548828 total time spend for step 5 : 29.889673471450806 step6:crop_condition Wed Feb 5 11:34:05 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 20 ! batch 1 Loaded 170 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 4869462 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738751647_3473826 we have uploaded 4 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.277738094329834 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738751649_3473826 we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.4977424144744873 we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 4869462 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738751651_3473826 we have uploaded 1 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.4648861885070801 we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738751652_3473826 we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.8976378440856934 we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! About to upload 8 photos upload in portfolio : 4869462 init cache_photo without model_param we have 8 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738751655_3473826 we have uploaded 8 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.1375954151153564 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 4869462 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738751658_3473826 we have uploaded 4 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.1119768619537354 we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738751660_3473826 we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.0728695392608643 we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 4869462 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738751662_3473826 we have uploaded 1 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.5062685012817383 we have finished the crop for the class : metal delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1332579049, 1332579041, 1332579037, 1332578979, 1332578949, 1332578909, 1332578907, 1332578904, 1332578882, 1332578875, 1332578871, 1332578867, 1332578866, 1332578863, 1332578856, 1332578830, 1332578813, 1332578798, 1332578634, 1332578632] Looping around the photos to save general results len do output : 27 /1334897216Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334897217Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334897218Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334897219Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334897223Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334897224Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334897225Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334897226Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334897228Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334897229Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334897230Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334897233Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334897234Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334897235Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334897236Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334897237Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334897238Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334897239Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334897240Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334897242Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334897243Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334897244Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334897245Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334897247Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334897248Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334897249Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334897256Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332579049', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332579041', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332579037', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578979', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578949', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578909', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578907', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578904', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578882', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578875', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578871', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578867', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578866', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578863', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578856', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578830', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578813', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578798', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578634', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578632', None, None, None, None, None, '2525467') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 101 time used for this insertion : 0.033072710037231445 save_final save missing photos in datou_result : time spend for datou_step_exec : 18.135307550430298 time spend to save output : 0.0339963436126709 total time spend for step 6 : 18.16930389404297 step7:ventilate_hashtags_in_portfolio Wed Feb 5 11:34:23 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 20038647 get user id for portfolio 20038647 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20038647 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','Teint_Dans_La_Masse','metal','mal_croppe','Carton_brun','Carton_gris','environnement','flou','autre_refus','plastique','kraft','cartonnette')) AND mptpi.`min_score`=0.5 To do To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20038647 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','Teint_Dans_La_Masse','metal','mal_croppe','Carton_brun','Carton_gris','environnement','flou','autre_refus','plastique','kraft','cartonnette')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/20038806,20038807,20038808,20038809,20038810,20038811,20038812,20038813,20038814,20038815,20038816,20038817?tags=Carton_brun,Teint_Dans_La_Masse,Carton_gris,environnement,metal,mal_croppe,papier,cartonnette,plastique,autre_refus,kraft,flou Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1332579049, 1332579041, 1332579037, 1332578979, 1332578949, 1332578909, 1332578907, 1332578904, 1332578882, 1332578875, 1332578871, 1332578867, 1332578866, 1332578863, 1332578856, 1332578830, 1332578813, 1332578798, 1332578634, 1332578632] Looping around the photos to save general results len do output : 1 /20038647. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332579049', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332579041', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332579037', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578979', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578949', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578909', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578907', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578904', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578882', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578875', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578871', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578867', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578866', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578863', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578856', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578830', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578813', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578798', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578634', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578632', None, None, None, None, None, '2525467') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 21 time used for this insertion : 0.01623082160949707 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.48063087463378906 time spend to save output : 0.016516923904418945 total time spend for step 7 : 0.497147798538208 step8:final Wed Feb 5 11:34:24 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1332579049: ('0.061483726550001255',), 1332579041: ('0.061483726550001255',), 1332579037: ('0.061483726550001255',), 1332578979: ('0.061483726550001255',), 1332578949: ('0.061483726550001255',), 1332578909: ('0.061483726550001255',), 1332578907: ('0.061483726550001255',), 1332578904: ('0.061483726550001255',), 1332578882: ('0.061483726550001255',), 1332578875: ('0.061483726550001255',), 1332578871: ('0.061483726550001255',), 1332578867: ('0.061483726550001255',), 1332578866: ('0.061483726550001255',), 1332578863: ('0.061483726550001255',), 1332578856: ('0.061483726550001255',), 1332578830: ('0.061483726550001255',), 1332578813: ('0.061483726550001255',), 1332578798: ('0.061483726550001255',), 1332578634: ('0.061483726550001255',), 1332578632: ('0.061483726550001255',)} new output for save of step final : {1332579049: ('0.061483726550001255',), 1332579041: ('0.061483726550001255',), 1332579037: ('0.061483726550001255',), 1332578979: ('0.061483726550001255',), 1332578949: ('0.061483726550001255',), 1332578909: ('0.061483726550001255',), 1332578907: ('0.061483726550001255',), 1332578904: ('0.061483726550001255',), 1332578882: ('0.061483726550001255',), 1332578875: ('0.061483726550001255',), 1332578871: ('0.061483726550001255',), 1332578867: ('0.061483726550001255',), 1332578866: ('0.061483726550001255',), 1332578863: ('0.061483726550001255',), 1332578856: ('0.061483726550001255',), 1332578830: ('0.061483726550001255',), 1332578813: ('0.061483726550001255',), 1332578798: ('0.061483726550001255',), 1332578634: ('0.061483726550001255',), 1332578632: ('0.061483726550001255',)} [1332579049, 1332579041, 1332579037, 1332578979, 1332578949, 1332578909, 1332578907, 1332578904, 1332578882, 1332578875, 1332578871, 1332578867, 1332578866, 1332578863, 1332578856, 1332578830, 1332578813, 1332578798, 1332578634, 1332578632] Looping around the photos to save general results len do output : 20 /1332579049.Didn't retrieve data . /1332579041.Didn't retrieve data . /1332579037.Didn't retrieve data . /1332578979.Didn't retrieve data . /1332578949.Didn't retrieve data . /1332578909.Didn't retrieve data . /1332578907.Didn't retrieve data . /1332578904.Didn't retrieve data . /1332578882.Didn't retrieve data . /1332578875.Didn't retrieve data . /1332578871.Didn't retrieve data . /1332578867.Didn't retrieve data . /1332578866.Didn't retrieve data . /1332578863.Didn't retrieve data . /1332578856.Didn't retrieve data . /1332578830.Didn't retrieve data . /1332578813.Didn't retrieve data . /1332578798.Didn't retrieve data . /1332578634.Didn't retrieve data . /1332578632.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332579049', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332579041', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332579037', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578979', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578949', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578909', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578907', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578904', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578882', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578875', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578871', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578867', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578866', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578863', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578856', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578830', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578813', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578798', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578634', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578632', None, None, None, None, None, '2525467') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 60 time used for this insertion : 0.0159609317779541 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.2763557434082031 time spend to save output : 0.0168304443359375 total time spend for step 8 : 0.2931861877441406 step9:velours_tree Wed Feb 5 11:34:24 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.02708578109741211 time spend to save output : 3.0517578125e-05 total time spend for step 9 : 0.02711629867553711 step10:send_mail_cod Wed Feb 5 11:34:24 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin in order to get the selector url, please entre the license of selector results_Auto_P20038647_05-02-2025_11_34_24.pdf 20038810 change filename to text .imagette200388101738751664 20038811 imagette200388111738751664 20038813 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette200388131738751664 20038814 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette200388141738751665 20038815 change filename to text .change filename to text .change filename to text .imagette200388151738751666 20038816 change filename to text .change filename to text .change filename to text .imagette200388161738751668 20038817 imagette200388171738751668 20038806 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette200388061738751668 20038807 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette200388071738751670 20038808 change filename to text .change filename to text .change filename to text .imagette200388081738751672 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20038647 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20038806,20038807,20038808,20038809,20038810,20038811,20038812,20038813,20038814,20038815,20038816,20038817?tags=Carton_brun,Teint_Dans_La_Masse,Carton_gris,environnement,metal,mal_croppe,papier,cartonnette,plastique,autre_refus,kraft,flou your option no_mail is active, we will not send the real mail to your client args[1332579049] : ('0.061483726550001255',) no score found for photo 1332579049 We are sending mail with results at report@fotonower.com args[1332579041] : ('0.061483726550001255',) no score found for photo 1332579041 We are sending mail with results at report@fotonower.com args[1332579037] : ('0.061483726550001255',) no score found for photo 1332579037 We are sending mail with results at report@fotonower.com args[1332578979] : ('0.061483726550001255',) no score found for photo 1332578979 We are sending mail with results at report@fotonower.com args[1332578949] : ('0.061483726550001255',) no score found for photo 1332578949 We are sending mail with results at report@fotonower.com args[1332578909] : ('0.061483726550001255',) no score found for photo 1332578909 We are sending mail with results at report@fotonower.com args[1332578907] : ('0.061483726550001255',) no score found for photo 1332578907 We are sending mail with results at report@fotonower.com args[1332578904] : ('0.061483726550001255',) no score found for photo 1332578904 We are sending mail with results at report@fotonower.com args[1332578882] : ('0.061483726550001255',) no score found for photo 1332578882 We are sending mail with results at report@fotonower.com args[1332578875] : ('0.061483726550001255',) no score found for photo 1332578875 We are sending mail with results at report@fotonower.com args[1332578871] : ('0.061483726550001255',) no score found for photo 1332578871 We are sending mail with results at report@fotonower.com args[1332578867] : ('0.061483726550001255',) no score found for photo 1332578867 We are sending mail with results at report@fotonower.com args[1332578866] : ('0.061483726550001255',) no score found for photo 1332578866 We are sending mail with results at report@fotonower.com args[1332578863] : ('0.061483726550001255',) no score found for photo 1332578863 We are sending mail with results at report@fotonower.com args[1332578856] : ('0.061483726550001255',) no score found for photo 1332578856 We are sending mail with results at report@fotonower.com args[1332578830] : ('0.061483726550001255',) no score found for photo 1332578830 We are sending mail with results at report@fotonower.com args[1332578813] : ('0.061483726550001255',) no score found for photo 1332578813 We are sending mail with results at report@fotonower.com args[1332578798] : ('0.061483726550001255',) no score found for photo 1332578798 We are sending mail with results at report@fotonower.com args[1332578634] : ('0.061483726550001255',) no score found for photo 1332578634 We are sending mail with results at report@fotonower.com args[1332578632] : ('0.061483726550001255',) no score found for photo 1332578632 We are sending mail with results at report@fotonower.com refus_total : 0.061483726550001255 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20038647 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1332572500,1332573990,1332578979,1332551137,1332551140,1332551157,1332551227,1332578909,1332551923,1332578798,1332552529,1332574574,1332552656,1332552662,1332578612,1332576485,1332571729,1332576616,1332576488,1332572360) Found this number of photos: 20 begin to download photo : 1332572500 begin to download photo : 1332551157 begin to download photo : 1332552529 begin to download photo : 1332576485 download finish for photo 1332552529 begin to download photo : 1332574574 download finish for photo 1332576485 begin to download photo : 1332571729 download finish for photo 1332572500 begin to download photo : 1332573990 download finish for photo 1332551157 begin to download photo : 1332551227 download finish for photo 1332573990 begin to download photo : 1332578979 download finish for photo 1332574574 begin to download photo : 1332552656 download finish for photo 1332551227 begin to download photo : 1332578909 download finish for photo 1332571729 begin to download photo : 1332576616 download finish for photo 1332578909 begin to download photo : 1332551923 download finish for photo 1332552656 begin to download photo : 1332552662 download finish for photo 1332578979 begin to download photo : 1332551137 download finish for photo 1332551923 begin to download photo : 1332578798 download finish for photo 1332576616 begin to download photo : 1332576488 download finish for photo 1332552662 begin to download photo : 1332578612 download finish for photo 1332551137 begin to download photo : 1332551140 download finish for photo 1332578798 download finish for photo 1332578612 download finish for photo 1332576488 begin to download photo : 1332572360 download finish for photo 1332551140 download finish for photo 1332572360 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20038647_05-02-2025_11_34_24.pdf results_Auto_P20038647_05-02-2025_11_34_24.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20038647_05-02-2025_11_34_24.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3459','20038647','results_Auto_P20038647_05-02-2025_11_34_24.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20038647_05-02-2025_11_34_24.pdf','pdf','','0.44','0.061483726550001255') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1332579049, 1332579041, 1332579037, 1332578979, 1332578949, 1332578909, 1332578907, 1332578904, 1332578882, 1332578875, 1332578871, 1332578867, 1332578866, 1332578863, 1332578856, 1332578830, 1332578813, 1332578798, 1332578634, 1332578632] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332579049', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332579041', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332579037', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578979', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578949', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578909', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578907', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578904', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578882', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578875', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578871', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578867', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578866', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578863', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578856', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578830', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578813', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578798', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578634', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578632', None, None, None, None, None, '2525467') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 20 time used for this insertion : 0.014405250549316406 save_final save missing photos in datou_result : time spend for datou_step_exec : 13.53952407836914 time spend to save output : 0.014660120010375977 total time spend for step 10 : 13.554184198379517 step11:split_time_score Wed Feb 5 11:34:38 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('07', 240),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 29012025 20038647 Nombre de photos uploadées : 240 / 23040 (1%) 29012025 20038647 Nombre de photos taguées (types de déchets): 0 / 240 (0%) 29012025 20038647 Nombre de photos taguées (volume) : 0 / 240 (0%) elapsed_time : load_data_split_time_score 2.6226043701171875e-06 elapsed_time : order_list_meta_photo_and_scores 7.62939453125e-06 ???????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.00941157341003418 elapsed_time : insert_dashboard_record_day_entry 0.03360557556152344 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.05964569986773992 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20038647_05-02-2025_11_34_24.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20038647 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20038647 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20038303 order by id desc limit 1 Qualite : 0.05453990257520215 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20036989_29-01-2025_08_12_24.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20036989 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20036989 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20037835 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20037561 order by id desc limit 1 Qualite : 0.029203380749725394 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20037563_29-01-2025_08_42_25.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20037563 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20037563 AND mptpi.`type`=3726 To do Qualite : 0.035880779652306634 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20038304_29-01-2025_09_22_44.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20038304 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20038304 AND mptpi.`type`=3726 To do Qualite : 0.04420808919501898 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20043199_04-02-2025_11_04_26.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20043199 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20043199 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20040004 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20040005 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20042811 order by id desc limit 1 Qualite : 0.04179473612560559 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20042812_29-01-2025_13_32_57.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20042812 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20042812 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20042942 order by id desc limit 1 Qualite : 0.050673850254881816 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20043200_04-02-2025_11_05_05.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20043200 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20043200 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20042552 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20042169 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20042170 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20042172 order by id desc limit 1 Qualite : 0.03250609913107722 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20043201_29-01-2025_14_11_08.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20043201 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20043201 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20043550 order by id desc limit 1 Qualite : 0.009347489222823262 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20043551_29-01-2025_14_41_40.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20043551 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20043551 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20043552 order by id desc limit 1 Qualite : 0.040548452513978116 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20044107_29-01-2025_15_32_02.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20044107 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20044107 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20044108 order by id desc limit 1 Qualite : 0.04088001272734784 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20044298_29-01-2025_15_27_01.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20044298 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20044298 AND mptpi.`type`=3726 To do Qualite : 0.025866735673049823 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20044671_29-01-2025_16_01_56.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20044671 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20044671 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20045174 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20045175 order by id desc limit 1 Qualite : 0.036707148122498 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20045657_29-01-2025_19_01_15.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20045657 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20045657 AND mptpi.`type`=3726 To do Qualite : 0.04449978208679873 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20045658_29-01-2025_16_51_25.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20045658 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20045658 AND mptpi.`type`=3726 To do Qualite : 0.06389528464627381 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20045933_29-01-2025_17_33_28.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20045933 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20045933 AND mptpi.`type`=3726 To do Qualite : 0.05614592496812481 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20046299_29-01-2025_18_02_54.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20046299 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20046299 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20047762 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20047763 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20047764 order by id desc limit 1 Qualite : 0.0513536384207915 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20047765_29-01-2025_19_33_11.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20047765 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20047765 AND mptpi.`type`=3726 To do Qualite : 0.030150108126073005 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20048159_29-01-2025_20_01_14.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20048159 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20048159 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20048160 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20048632 order by id desc limit 1 Qualite : 0.02217739158266012 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20048633_29-01-2025_20_21_56.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20048633 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20048633 AND mptpi.`type`=3726 To do Qualite : 0.04372363561039585 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20048832_29-01-2025_20_27_01.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20048832 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20048832 AND mptpi.`type`=3726 To do Qualite : 0.007334799965609816 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20049145_29-01-2025_21_00_41.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20049145 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20049145 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20105092 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20066611 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20066612 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20066613 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20066615 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20066616 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20066617 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20105093 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20066619 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20066620 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20066621 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20105118 order by id desc limit 1 NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'29012025': {'nb_upload': 240, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1332579049, 1332579041, 1332579037, 1332578979, 1332578949, 1332578909, 1332578907, 1332578904, 1332578882, 1332578875, 1332578871, 1332578867, 1332578866, 1332578863, 1332578856, 1332578830, 1332578813, 1332578798, 1332578634, 1332578632] Looping around the photos to save general results len do output : 1 /20038647Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332579049', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332579041', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332579037', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578979', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578949', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578909', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578907', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578904', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578882', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578875', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578871', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578867', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578866', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578863', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578856', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578830', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578813', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578798', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578634', None, None, None, None, None, '2525467') ('3459', None, None, None, None, None, None, None, '2525467') ('3459', None, '1332578632', None, None, None, None, None, '2525467') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 21 time used for this insertion : 0.014701128005981445 save_final save missing photos in datou_result : time spend for datou_step_exec : 42.71431374549866 time spend to save output : 0.014894485473632812 total time spend for step 11 : 42.72920823097229 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 84.87user 28.26system 3:00.95elapsed 62%CPU (0avgtext+0avgdata 3129444maxresident)k 280inputs+66256outputs (17major+2299892minor)pagefaults 0swaps