python /home/admin/mtr/script_for_cron.py -j default -m 20 -a 'python3 ~/workarea/git/Velours/python/prod/datou.py -j batch_current -C 2533239' -s traitement_3459 -M 0 -S 0 -U 100,80,95 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/home/admin/workarea/git/apy', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3455466 load datou : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : step 0 init_dummy_multi_datou is not linked in the step_by_step architecture ! WARNING : step 1294 init_dummy_multi_datou is not linked in the step_by_step architecture ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : (photo_id, hashtag_id, score_max) was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3459, datou_cur_ids : ['2533239'] with mtr_portfolio_ids : ['20105213'] and first list_photo_ids : [] new path : /proc/3455466/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, thcl, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 20 list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 20 ; length of list_pids : 20 ; length of list_args : 20 time to download the photos : 3.0273280143737793 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Wed Feb 5 11:25:47 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 3000 l 3632 free memory gpu now : 2105 wait 20 seconds l 3637 free memory gpu now : 2105 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-05 11:26:10.431701: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-05 11:26:10.459087: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-05 11:26:10.460715: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fcd20000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-05 11:26:10.460759: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-05 11:26:10.463348: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-05 11:26:10.616659: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x33208a60 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-05 11:26:10.616714: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-05 11:26:10.618087: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-05 11:26:10.618475: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-05 11:26:10.621365: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-05 11:26:10.623894: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-05 11:26:10.624354: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-05 11:26:10.627309: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-05 11:26:10.628685: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-05 11:26:10.632622: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-05 11:26:10.633988: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-05 11:26:10.634046: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-05 11:26:10.634778: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-05 11:26:10.634806: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-05 11:26:10.634815: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-05 11:26:10.636138: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 8864 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-05 11:26:10.914583: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-05 11:26:10.914712: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-05 11:26:10.914742: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-05 11:26:10.914767: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-05 11:26:10.914792: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-05 11:26:10.914817: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-05 11:26:10.914841: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-05 11:26:10.914867: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-05 11:26:10.916826: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-05 11:26:10.918263: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-05 11:26:10.918293: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-05 11:26:10.918308: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-05 11:26:10.918322: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-05 11:26:10.918336: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-05 11:26:10.918350: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-05 11:26:10.918364: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-05 11:26:10.918378: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-05 11:26:10.919567: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-05 11:26:10.919602: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-05 11:26:10.919611: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-05 11:26:10.919619: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-05 11:26:10.920854: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 8864 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-05 11:26:18.525033: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-05 11:26:18.704089: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 20 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 7.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 2 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 24.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 6.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 1.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 9.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 3.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 1.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 Detection mask done ! Trying to reset tf kernel 3456236 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5485 tf kernel not reseted sub process len(results) : 20 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 20 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10774 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.002574443817138672 nb_pixel_total : 125678 time to create 1 rle with old method : 0.13050246238708496 length of segment : 681 time for calcul the mask position with numpy : 0.000118255615234375 nb_pixel_total : 2854 time to create 1 rle with old method : 0.003249645233154297 length of segment : 52 time for calcul the mask position with numpy : 0.006424665451049805 nb_pixel_total : 403347 time to create 1 rle with new method : 0.022284984588623047 length of segment : 998 time for calcul the mask position with numpy : 0.0018961429595947266 nb_pixel_total : 104622 time to create 1 rle with old method : 0.11246418952941895 length of segment : 687 time for calcul the mask position with numpy : 0.02148294448852539 nb_pixel_total : 863713 time to create 1 rle with new method : 0.050547122955322266 length of segment : 1611 time for calcul the mask position with numpy : 0.0030295848846435547 nb_pixel_total : 213254 time to create 1 rle with new method : 0.008131265640258789 length of segment : 1043 time for calcul the mask position with numpy : 0.0004162788391113281 nb_pixel_total : 25055 time to create 1 rle with old method : 0.027089357376098633 length of segment : 170 time for calcul the mask position with numpy : 0.0001285076141357422 nb_pixel_total : 4748 time to create 1 rle with old method : 0.00518035888671875 length of segment : 125 time for calcul the mask position with numpy : 0.0007634162902832031 nb_pixel_total : 35858 time to create 1 rle with old method : 0.038564205169677734 length of segment : 396 time for calcul the mask position with numpy : 0.0225064754486084 nb_pixel_total : 1253586 time to create 1 rle with new method : 0.05001187324523926 length of segment : 2715 time for calcul the mask position with numpy : 0.002960205078125 nb_pixel_total : 127678 time to create 1 rle with old method : 0.13387727737426758 length of segment : 570 time for calcul the mask position with numpy : 0.001031637191772461 nb_pixel_total : 45853 time to create 1 rle with old method : 0.05084037780761719 length of segment : 276 time for calcul the mask position with numpy : 0.016078948974609375 nb_pixel_total : 1050189 time to create 1 rle with new method : 0.03387117385864258 length of segment : 1403 time for calcul the mask position with numpy : 0.003421783447265625 nb_pixel_total : 310245 time to create 1 rle with new method : 0.007517337799072266 length of segment : 801 time for calcul the mask position with numpy : 0.0003314018249511719 nb_pixel_total : 6240 time to create 1 rle with old method : 0.007297515869140625 length of segment : 98 time for calcul the mask position with numpy : 0.0020606517791748047 nb_pixel_total : 90982 time to create 1 rle with old method : 0.10103535652160645 length of segment : 425 time for calcul the mask position with numpy : 0.007805824279785156 nb_pixel_total : 384843 time to create 1 rle with new method : 0.023142576217651367 length of segment : 1157 time for calcul the mask position with numpy : 0.0008153915405273438 nb_pixel_total : 26632 time to create 1 rle with old method : 0.030161619186401367 length of segment : 255 time for calcul the mask position with numpy : 0.004531383514404297 nb_pixel_total : 199801 time to create 1 rle with new method : 0.01171565055847168 length of segment : 1002 time for calcul the mask position with numpy : 0.0004620552062988281 nb_pixel_total : 11963 time to create 1 rle with old method : 0.014185190200805664 length of segment : 124 time for calcul the mask position with numpy : 0.0006091594696044922 nb_pixel_total : 38090 time to create 1 rle with old method : 0.042265892028808594 length of segment : 117 time for calcul the mask position with numpy : 0.017188310623168945 nb_pixel_total : 1006563 time to create 1 rle with new method : 0.03465747833251953 length of segment : 1512 time for calcul the mask position with numpy : 0.002162456512451172 nb_pixel_total : 85770 time to create 1 rle with old method : 0.09348344802856445 length of segment : 284 time for calcul the mask position with numpy : 0.002849578857421875 nb_pixel_total : 146379 time to create 1 rle with old method : 0.16576671600341797 length of segment : 547 time for calcul the mask position with numpy : 0.00024962425231933594 nb_pixel_total : 4010 time to create 1 rle with old method : 0.00478053092956543 length of segment : 91 time for calcul the mask position with numpy : 0.0006690025329589844 nb_pixel_total : 37601 time to create 1 rle with old method : 0.0440821647644043 length of segment : 116 time for calcul the mask position with numpy : 0.019551753997802734 nb_pixel_total : 1151326 time to create 1 rle with new method : 0.03749203681945801 length of segment : 1673 time for calcul the mask position with numpy : 0.00019788742065429688 nb_pixel_total : 10912 time to create 1 rle with old method : 0.01258540153503418 length of segment : 136 time for calcul the mask position with numpy : 0.00022554397583007812 nb_pixel_total : 6819 time to create 1 rle with old method : 0.008089065551757812 length of segment : 113 time for calcul the mask position with numpy : 0.0007796287536621094 nb_pixel_total : 29383 time to create 1 rle with old method : 0.032670021057128906 length of segment : 263 time for calcul the mask position with numpy : 0.0006818771362304688 nb_pixel_total : 45512 time to create 1 rle with old method : 0.05115485191345215 length of segment : 195 time for calcul the mask position with numpy : 0.001911163330078125 nb_pixel_total : 115107 time to create 1 rle with old method : 0.126662015914917 length of segment : 370 time for calcul the mask position with numpy : 0.008369922637939453 nb_pixel_total : 465481 time to create 1 rle with new method : 0.023241758346557617 length of segment : 1598 time for calcul the mask position with numpy : 0.014682531356811523 nb_pixel_total : 849141 time to create 1 rle with new method : 0.035993337631225586 length of segment : 2118 time for calcul the mask position with numpy : 0.0005903244018554688 nb_pixel_total : 21028 time to create 1 rle with old method : 0.02463817596435547 length of segment : 122 time for calcul the mask position with numpy : 0.0022919178009033203 nb_pixel_total : 160697 time to create 1 rle with new method : 0.005594968795776367 length of segment : 690 time for calcul the mask position with numpy : 0.0036880970001220703 nb_pixel_total : 256459 time to create 1 rle with new method : 0.009866714477539062 length of segment : 739 time for calcul the mask position with numpy : 0.0002257823944091797 nb_pixel_total : 15306 time to create 1 rle with old method : 0.018318891525268555 length of segment : 81 time for calcul the mask position with numpy : 0.013350963592529297 nb_pixel_total : 676068 time to create 1 rle with new method : 0.03493380546569824 length of segment : 1285 time for calcul the mask position with numpy : 0.0010848045349121094 nb_pixel_total : 28467 time to create 1 rle with old method : 0.03135943412780762 length of segment : 485 time for calcul the mask position with numpy : 0.002195596694946289 nb_pixel_total : 78932 time to create 1 rle with old method : 0.08733630180358887 length of segment : 651 time for calcul the mask position with numpy : 0.000812530517578125 nb_pixel_total : 28407 time to create 1 rle with old method : 0.03693890571594238 length of segment : 291 time for calcul the mask position with numpy : 0.000308990478515625 nb_pixel_total : 6722 time to create 1 rle with old method : 0.007653474807739258 length of segment : 162 time for calcul the mask position with numpy : 0.0017895698547363281 nb_pixel_total : 85910 time to create 1 rle with old method : 0.100250244140625 length of segment : 604 time for calcul the mask position with numpy : 0.014961004257202148 nb_pixel_total : 780727 time to create 1 rle with new method : 0.03592324256896973 length of segment : 1513 time for calcul the mask position with numpy : 0.0001049041748046875 nb_pixel_total : 2147 time to create 1 rle with old method : 0.0030243396759033203 length of segment : 47 time for calcul the mask position with numpy : 0.00018930435180664062 nb_pixel_total : 6540 time to create 1 rle with old method : 0.007627725601196289 length of segment : 164 time for calcul the mask position with numpy : 0.0004475116729736328 nb_pixel_total : 14379 time to create 1 rle with old method : 0.01685476303100586 length of segment : 153 time for calcul the mask position with numpy : 0.002398252487182617 nb_pixel_total : 99761 time to create 1 rle with old method : 0.11559700965881348 length of segment : 443 time for calcul the mask position with numpy : 0.006407976150512695 nb_pixel_total : 320350 time to create 1 rle with new method : 0.01645493507385254 length of segment : 1530 time for calcul the mask position with numpy : 0.0008771419525146484 nb_pixel_total : 16203 time to create 1 rle with old method : 0.023763179779052734 length of segment : 161 time for calcul the mask position with numpy : 0.0001571178436279297 nb_pixel_total : 1312 time to create 1 rle with old method : 0.0016949176788330078 length of segment : 60 time for calcul the mask position with numpy : 0.009207963943481445 nb_pixel_total : 371150 time to create 1 rle with new method : 0.024697542190551758 length of segment : 1601 time for calcul the mask position with numpy : 0.013734579086303711 nb_pixel_total : 806587 time to create 1 rle with new method : 0.03351449966430664 length of segment : 2042 time for calcul the mask position with numpy : 0.0014562606811523438 nb_pixel_total : 106721 time to create 1 rle with old method : 0.12359356880187988 length of segment : 292 time for calcul the mask position with numpy : 0.000873565673828125 nb_pixel_total : 61959 time to create 1 rle with old method : 0.09420919418334961 length of segment : 249 time for calcul the mask position with numpy : 0.00030612945556640625 nb_pixel_total : 6200 time to create 1 rle with old method : 0.007195949554443359 length of segment : 135 time for calcul the mask position with numpy : 0.008052587509155273 nb_pixel_total : 258629 time to create 1 rle with new method : 0.022241592407226562 length of segment : 935 time for calcul the mask position with numpy : 0.0018925666809082031 nb_pixel_total : 106462 time to create 1 rle with old method : 0.11766982078552246 length of segment : 555 time for calcul the mask position with numpy : 0.0009801387786865234 nb_pixel_total : 62820 time to create 1 rle with old method : 0.07087993621826172 length of segment : 277 time for calcul the mask position with numpy : 0.01455998420715332 nb_pixel_total : 954506 time to create 1 rle with new method : 0.02806687355041504 length of segment : 2000 time for calcul the mask position with numpy : 0.0005726814270019531 nb_pixel_total : 33493 time to create 1 rle with old method : 0.0357670783996582 length of segment : 241 time for calcul the mask position with numpy : 0.0029134750366210938 nb_pixel_total : 205198 time to create 1 rle with new method : 0.008190631866455078 length of segment : 802 time for calcul the mask position with numpy : 0.00030517578125 nb_pixel_total : 1619 time to create 1 rle with old method : 0.0035240650177001953 length of segment : 56 time for calcul the mask position with numpy : 0.014993429183959961 nb_pixel_total : 826766 time to create 1 rle with new method : 0.030203580856323242 length of segment : 1134 time for calcul the mask position with numpy : 0.0006456375122070312 nb_pixel_total : 36317 time to create 1 rle with old method : 0.040726423263549805 length of segment : 219 time for calcul the mask position with numpy : 0.0026235580444335938 nb_pixel_total : 172973 time to create 1 rle with new method : 0.006738185882568359 length of segment : 794 time for calcul the mask position with numpy : 0.0017976760864257812 nb_pixel_total : 60827 time to create 1 rle with old method : 0.06824445724487305 length of segment : 406 time for calcul the mask position with numpy : 0.0005791187286376953 nb_pixel_total : 41848 time to create 1 rle with old method : 0.04772758483886719 length of segment : 284 time for calcul the mask position with numpy : 0.01092386245727539 nb_pixel_total : 548654 time to create 1 rle with new method : 0.027872562408447266 length of segment : 1339 time for calcul the mask position with numpy : 0.012312889099121094 nb_pixel_total : 730514 time to create 1 rle with new method : 0.030300378799438477 length of segment : 1340 time for calcul the mask position with numpy : 0.00012993812561035156 nb_pixel_total : 3788 time to create 1 rle with old method : 0.004469394683837891 length of segment : 91 time for calcul the mask position with numpy : 0.0010406970977783203 nb_pixel_total : 60243 time to create 1 rle with old method : 0.07333970069885254 length of segment : 395 time for calcul the mask position with numpy : 0.0011975765228271484 nb_pixel_total : 57343 time to create 1 rle with old method : 0.06403827667236328 length of segment : 383 time for calcul the mask position with numpy : 0.0011985301971435547 nb_pixel_total : 85503 time to create 1 rle with old method : 0.09645271301269531 length of segment : 275 time spent for convertir_results : 11.047407150268555 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 87 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 48748 save missing photos in datou_result : time spend for datou_step_exec : 52.23350691795349 time spend to save output : 3.2461748123168945 total time spend for step 1 : 55.479681730270386 step2:crop_condition Wed Feb 5 11:26:43 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 20 ! batch 1 Loaded 87 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 13 About to insert : list_path_to_insert length 13 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1333154110, 1333154091, 1333154077, 1333153911, 1333153906, 1333153900, 1333153895, 1333153887, 1333153884, 1333153878, 1333153875, 1333153873, 1333153869, 1333153864, 1333153858, 1333153729, 1333153728, 1333153723, 1333153718, 1333153713] Looping around the photos to save general results len do output : 29 /-3655803496Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655380347Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655380352Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655803518Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655803531Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655803541Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655803504Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655803516Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655803538Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655380341Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655380344Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655380350Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655380355Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655380358Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655380356Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655380361Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655380368Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655803507Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655803513Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655803525Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655803529Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655803542Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655803515Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655380339Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655380353Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655380360Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655380369Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655803512Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3655803544Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333154110', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333154091', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333154077', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153911', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153906', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153900', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153895', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153887', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153884', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153878', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153875', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153873', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153869', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153864', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153858', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153729', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153728', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153723', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153718', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153713', None, None, None, None, None, '2533239') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 107 time used for this insertion : 0.025306224822998047 save_final save missing photos in datou_result : time spend for datou_step_exec : 8.902438402175903 time spend to save output : 0.026906490325927734 total time spend for step 2 : 8.929344892501831 step3:thcl Wed Feb 5 11:26:52 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 2 we are using the classfication for multi_thcl [2456, 2868] time to import caffe and check if the image exist : 0.006369829177856445 time to convert the images to numpy array : 0.0076045989990234375 time to import caffe and check if the image exist : 0.008950471878051758 time to convert the images to numpy array : 0.009644508361816406 time to import caffe and check if the image exist : 0.004229068756103516 time to convert the images to numpy array : 0.024758100509643555 time to import caffe and check if the image exist : 0.007194995880126953 time to convert the images to numpy array : 0.026220083236694336 time to import caffe and check if the image exist : 0.006848335266113281 time to convert the images to numpy array : 0.030231475830078125 time to import caffe and check if the image exist : 0.00785207748413086 time to convert the images to numpy array : 0.029447555541992188 time to import caffe and check if the image exist : 0.003635883331298828 time to convert the images to numpy array : 0.035456180572509766 time to import caffe and check if the image exist : 0.004545688629150391 time to convert the images to numpy array : 0.03632092475891113 time to import caffe and check if the image exist : 0.004464626312255859 time to convert the images to numpy array : 0.040181636810302734 time to import caffe and check if the image exist : 0.00914907455444336 time to convert the images to numpy array : 0.040366172790527344 total time to convert the images to numpy array : 0.29175472259521484 list photo_ids error: [] list photo_ids correct : [-3655803496, -3655380347, -3655380352, -3655380353, -3655380360, -3655380369, -3655803512, -3655803544, -3655803542, -3655803515, -3655380339, -3655380341, -3655380344, -3655380350, -3655803518, -3655803531, -3655803541, -3655380361, -3655380368, -3655803507, -3655803504, -3655803516, -3655803538, -3655803513, -3655803525, -3655803529, -3655380355, -3655380358, -3655380356] number of photos to traite : 29 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 8455 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/:/home/admin/workarea/git/apy/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10334 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 0.24618959426879883 time used to do the prediction : 0.0962681770324707 we don't save the descriptors for this thcl 2456 tagging for thcl : 2868 To do loadFromThcl(), then load ParamDescType : thcl2868 thcls : [{'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5288 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) To loadFromThcl() : net_5288 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10332 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_papier_nantes_300421 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_papier_nantes_300421 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_papier_nantes_300421 /data/models_weight/learn_papier_nantes_300421/caffemodel size_local : 44791983 size in s3 : 44791983 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:13 caffemodel already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/deploy.prototxt size_local : 17255 size in s3 : 17255 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:12 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:14 mean.npy already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/synset_words.txt size_local : 331 size in s3 : 331 create time local : 2021-08-09 05:56:00 create time in s3 : 2021-08-06 19:22:12 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/:/home/admin/workarea/git/apy/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_papier_nantes_300421/deploy.prototxt caffemodel_filename : /data/models_weight/learn_papier_nantes_300421/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10332 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 0.2766432762145996 time used to do the prediction : 0.06700801849365234 we don't save the descriptors for this thcl 2868 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 1.5497207641601562e-05 save missing photos in datou_result : time spend for datou_step_exec : 7.520252466201782 time spend to save output : 0.0003452301025390625 total time spend for step 3 : 7.520597696304321 step4:merge_mask_thcl_custom Wed Feb 5 11:26:59 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 87 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1333154110, 1333154091, 1333154077, 1333153911, 1333153906, 1333153900, 1333153895, 1333153887, 1333153884, 1333153878, 1333153875, 1333153873, 1333153869, 1333153864, 1333153858, 1333153729, 1333153728, 1333153723, 1333153718, 1333153713] Looping around the photos to save general results len do output : 20 /1333154110Didn't retrieve data .Didn't retrieve data . /1333154091Didn't retrieve data .Didn't retrieve data . /1333154077Didn't retrieve data .Didn't retrieve data . /1333153911Didn't retrieve data .Didn't retrieve data . /1333153906Didn't retrieve data .Didn't retrieve data . /1333153900Didn't retrieve data .Didn't retrieve data . /1333153895Didn't retrieve data .Didn't retrieve data . /1333153887Didn't retrieve data .Didn't retrieve data . /1333153884Didn't retrieve data .Didn't retrieve data . /1333153878Didn't retrieve data .Didn't retrieve data . /1333153875Didn't retrieve data .Didn't retrieve data . /1333153873Didn't retrieve data .Didn't retrieve data . /1333153869Didn't retrieve data .Didn't retrieve data . /1333153864Didn't retrieve data .Didn't retrieve data . /1333153858Didn't retrieve data .Didn't retrieve data . /1333153729Didn't retrieve data .Didn't retrieve data . /1333153728Didn't retrieve data .Didn't retrieve data . /1333153723Didn't retrieve data .Didn't retrieve data . /1333153718Didn't retrieve data .Didn't retrieve data . /1333153713Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333154110', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333154091', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333154077', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153911', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153906', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153900', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153895', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153887', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153884', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153878', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153875', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153873', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153869', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153864', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153858', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153729', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153728', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153723', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153718', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153713', None, None, None, None, None, '2533239') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 60 time used for this insertion : 2.263331174850464 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.11029458045959473 time spend to save output : 2.264198064804077 total time spend for step 4 : 2.374492645263672 step5:rle_unique_nms_with_priority Wed Feb 5 11:27:01 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 6 nb_hashtags : 2 time to prepare the origin masks : 0.5702905654907227 time for calcul the mask position with numpy : 0.019513845443725586 nb_pixel_total : 1470275 time to create 1 rle with new method : 0.04362845420837402 time for calcul the mask position with numpy : 0.006373405456542969 nb_pixel_total : 419 time to create 1 rle with old method : 0.0008661746978759766 time for calcul the mask position with numpy : 0.006461381912231445 nb_pixel_total : 48785 time to create 1 rle with old method : 0.052739858627319336 time for calcul the mask position with numpy : 0.006224870681762695 nb_pixel_total : 22685 time to create 1 rle with old method : 0.025090932846069336 time for calcul the mask position with numpy : 0.005904197692871094 nb_pixel_total : 2812 time to create 1 rle with old method : 0.003328084945678711 time for calcul the mask position with numpy : 0.010162591934204102 nb_pixel_total : 403583 time to create 1 rle with new method : 0.03973054885864258 time for calcul the mask position with numpy : 0.007302761077880859 nb_pixel_total : 125041 time to create 1 rle with old method : 0.132568359375 create new chi : 0.3684549331665039 time to delete rle : 0.016979694366455078 batch 1 Loaded 9 chid ids of type : 3726 Number RLEs to save : 7079 TO DO : save crop sub photo not yet done ! save time : 0.6662442684173584 nb_obj : 6 nb_hashtags : 3 time to prepare the origin masks : 0.9277794361114502 time for calcul the mask position with numpy : 0.015170097351074219 nb_pixel_total : 1081697 time to create 1 rle with new method : 0.04622220993041992 time for calcul the mask position with numpy : 0.007258176803588867 nb_pixel_total : 1387 time to create 1 rle with old method : 0.0022611618041992188 time for calcul the mask position with numpy : 0.007338047027587891 nb_pixel_total : 78228 time to create 1 rle with old method : 0.08845353126525879 time for calcul the mask position with numpy : 0.007353544235229492 nb_pixel_total : 25056 time to create 1 rle with old method : 0.02817678451538086 time for calcul the mask position with numpy : 0.00711512565612793 nb_pixel_total : 3000 time to create 1 rle with old method : 0.0050122737884521484 time for calcul the mask position with numpy : 0.013375282287597656 nb_pixel_total : 848562 time to create 1 rle with new method : 0.04420781135559082 time for calcul the mask position with numpy : 0.0072176456451416016 nb_pixel_total : 35670 time to create 1 rle with old method : 0.04177117347717285 create new chi : 0.32993531227111816 time to delete rle : 0.0008478164672851562 batch 1 Loaded 9 chid ids of type : 3726 Number RLEs to save : 7377 TO DO : save crop sub photo not yet done ! save time : 0.439241886138916 nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 0.7097642421722412 time for calcul the mask position with numpy : 0.014233827590942383 nb_pixel_total : 713999 time to create 1 rle with new method : 0.03920483589172363 time for calcul the mask position with numpy : 0.0064203739166259766 nb_pixel_total : 38604 time to create 1 rle with old method : 0.04190802574157715 time for calcul the mask position with numpy : 0.006347179412841797 nb_pixel_total : 38590 time to create 1 rle with old method : 0.04279685020446777 time for calcul the mask position with numpy : 0.00742030143737793 nb_pixel_total : 6224 time to create 1 rle with old method : 0.01002359390258789 time for calcul the mask position with numpy : 0.014901399612426758 nb_pixel_total : 1276183 time to create 1 rle with new method : 0.04150032997131348 create new chi : 0.23198223114013672 time to delete rle : 0.0008068084716796875 batch 1 Loaded 8 chid ids of type : 3726 Number RLEs to save : 8807 TO DO : save crop sub photo not yet done ! save time : 0.507847785949707 nb_obj : 3 nb_hashtags : 1 time to prepare the origin masks : 0.2378098964691162 time for calcul the mask position with numpy : 0.014433145523071289 nb_pixel_total : 957244 time to create 1 rle with new method : 0.04039955139160156 time for calcul the mask position with numpy : 0.010682821273803711 nb_pixel_total : 64345 time to create 1 rle with old method : 0.07585453987121582 time for calcul the mask position with numpy : 0.00669097900390625 nb_pixel_total : 1993 time to create 1 rle with old method : 0.0036940574645996094 time for calcul the mask position with numpy : 0.019141674041748047 nb_pixel_total : 1050018 time to create 1 rle with new method : 0.039238929748535156 create new chi : 0.2159574031829834 time to delete rle : 0.0005402565002441406 batch 1 Loaded 4 chid ids of type : 3726 Number RLEs to save : 5087 TO DO : save crop sub photo not yet done ! save time : 0.30886411666870117 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.24140572547912598 time for calcul the mask position with numpy : 0.02209639549255371 nb_pixel_total : 1672129 time to create 1 rle with new method : 0.03901529312133789 time for calcul the mask position with numpy : 0.006318807601928711 nb_pixel_total : 6221 time to create 1 rle with old method : 0.006840944290161133 time for calcul the mask position with numpy : 0.008118391036987305 nb_pixel_total : 304821 time to create 1 rle with new method : 0.029402494430541992 time for calcul the mask position with numpy : 0.0066890716552734375 nb_pixel_total : 90429 time to create 1 rle with old method : 0.09905624389648438 create new chi : 0.218780517578125 time to delete rle : 0.00042319297790527344 batch 1 Loaded 4 chid ids of type : 3726 Number RLEs to save : 4113 TO DO : save crop sub photo not yet done ! save time : 0.336592435836792 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.28046703338623047 time for calcul the mask position with numpy : 0.01900959014892578 nb_pixel_total : 1844243 time to create 1 rle with new method : 0.038027048110961914 time for calcul the mask position with numpy : 0.0066111087799072266 nb_pixel_total : 26454 time to create 1 rle with old method : 0.02965545654296875 time for calcul the mask position with numpy : 0.006232023239135742 nb_pixel_total : 3052 time to create 1 rle with old method : 0.0042383670806884766 time for calcul the mask position with numpy : 0.007287263870239258 nb_pixel_total : 199851 time to create 1 rle with new method : 0.037168025970458984 create new chi : 0.15622353553771973 time to delete rle : 0.0004968643188476562 batch 1 Loaded 5 chid ids of type : 3726 Number RLEs to save : 4608 TO DO : save crop sub photo not yet done ! save time : 0.29366230964660645 nb_obj : 7 nb_hashtags : 3 time to prepare the origin masks : 0.5404415130615234 time for calcul the mask position with numpy : 0.012710809707641602 nb_pixel_total : 853681 time to create 1 rle with new method : 0.04252982139587402 time for calcul the mask position with numpy : 0.005933523178100586 nb_pixel_total : 3993 time to create 1 rle with old method : 0.004338979721069336 time for calcul the mask position with numpy : 0.012642383575439453 nb_pixel_total : 933341 time to create 1 rle with new method : 0.03711199760437012 time for calcul the mask position with numpy : 0.006222963333129883 nb_pixel_total : 1141 time to create 1 rle with old method : 0.0014455318450927734 time for calcul the mask position with numpy : 0.006791114807128906 nb_pixel_total : 146231 time to create 1 rle with old method : 0.15385746955871582 time for calcul the mask position with numpy : 0.006436824798583984 nb_pixel_total : 37925 time to create 1 rle with old method : 0.04023933410644531 time for calcul the mask position with numpy : 0.0066051483154296875 nb_pixel_total : 85413 time to create 1 rle with old method : 0.08980393409729004 time for calcul the mask position with numpy : 0.006177425384521484 nb_pixel_total : 11875 time to create 1 rle with old method : 0.012865066528320312 create new chi : 0.450275182723999 time to delete rle : 0.0005424022674560547 batch 1 Loaded 8 chid ids of type : 3726 Number RLEs to save : 5764 TO DO : save crop sub photo not yet done ! save time : 1.432197093963623 nb_obj : 3 nb_hashtags : 3 time to prepare the origin masks : 0.2919025421142578 time for calcul the mask position with numpy : 0.01338052749633789 nb_pixel_total : 914669 time to create 1 rle with new method : 0.03348040580749512 time for calcul the mask position with numpy : 0.006625175476074219 nb_pixel_total : 6812 time to create 1 rle with old method : 0.007862567901611328 time for calcul the mask position with numpy : 0.006211280822753906 nb_pixel_total : 1939 time to create 1 rle with old method : 0.002267599105834961 time for calcul the mask position with numpy : 0.015999317169189453 nb_pixel_total : 1150180 time to create 1 rle with new method : 0.03580164909362793 create new chi : 0.1246182918548584 time to delete rle : 0.00047898292541503906 batch 1 Loaded 4 chid ids of type : 3726 Number RLEs to save : 4787 TO DO : save crop sub photo not yet done ! save time : 0.3318595886230469 nb_obj : 5 nb_hashtags : 2 time to prepare the origin masks : 0.4464435577392578 time for calcul the mask position with numpy : 0.019640207290649414 nb_pixel_total : 1401585 time to create 1 rle with new method : 0.04410123825073242 time for calcul the mask position with numpy : 0.007234811782836914 nb_pixel_total : 2645 time to create 1 rle with old method : 0.003968238830566406 time for calcul the mask position with numpy : 0.01105809211730957 nb_pixel_total : 479644 time to create 1 rle with new method : 0.04035639762878418 time for calcul the mask position with numpy : 0.007312774658203125 nb_pixel_total : 115006 time to create 1 rle with old method : 0.1237485408782959 time for calcul the mask position with numpy : 0.006508588790893555 nb_pixel_total : 45379 time to create 1 rle with old method : 0.050730228424072266 time for calcul the mask position with numpy : 0.007146358489990234 nb_pixel_total : 29341 time to create 1 rle with old method : 0.032495737075805664 create new chi : 0.35918736457824707 time to delete rle : 0.000606536865234375 batch 1 Loaded 8 chid ids of type : 3726 Number RLEs to save : 6197 TO DO : save crop sub photo not yet done ! save time : 0.4496769905090332 nb_obj : 6 nb_hashtags : 3 time to prepare the origin masks : 0.968442440032959 time for calcul the mask position with numpy : 0.015086174011230469 nb_pixel_total : 947446 time to create 1 rle with new method : 0.04456686973571777 time for calcul the mask position with numpy : 0.006594419479370117 nb_pixel_total : 5769 time to create 1 rle with old method : 0.0066242218017578125 time for calcul the mask position with numpy : 0.007708311080932617 nb_pixel_total : 155660 time to create 1 rle with new method : 0.0364840030670166 time for calcul the mask position with numpy : 0.007353305816650391 nb_pixel_total : 76472 time to create 1 rle with old method : 0.08558440208435059 time for calcul the mask position with numpy : 0.007162809371948242 nb_pixel_total : 15263 time to create 1 rle with old method : 0.016737937927246094 time for calcul the mask position with numpy : 0.006874561309814453 nb_pixel_total : 6122 time to create 1 rle with old method : 0.008410930633544922 time for calcul the mask position with numpy : 0.012582778930664062 nb_pixel_total : 866868 time to create 1 rle with new method : 0.03709602355957031 create new chi : 0.302654504776001 time to delete rle : 0.0007472038269042969 batch 1 Loaded 9 chid ids of type : 3726 Number RLEs to save : 8841 TO DO : save crop sub photo not yet done ! save time : 0.6281716823577881 nb_obj : 4 nb_hashtags : 1 time to prepare the origin masks : 0.39124298095703125 time for calcul the mask position with numpy : 0.01468801498413086 nb_pixel_total : 1272413 time to create 1 rle with new method : 0.032022953033447266 time for calcul the mask position with numpy : 0.006132364273071289 nb_pixel_total : 25206 time to create 1 rle with old method : 0.02717280387878418 time for calcul the mask position with numpy : 0.006656646728515625 nb_pixel_total : 72862 time to create 1 rle with old method : 0.07735204696655273 time for calcul the mask position with numpy : 0.006226778030395508 nb_pixel_total : 28362 time to create 1 rle with old method : 0.030818462371826172 time for calcul the mask position with numpy : 0.011188030242919922 nb_pixel_total : 674757 time to create 1 rle with new method : 0.027306079864501953 create new chi : 0.24146032333374023 time to delete rle : 0.0005714893341064453 batch 1 Loaded 5 chid ids of type : 3726 Number RLEs to save : 6368 TO DO : save crop sub photo not yet done ! save time : 0.3888096809387207 nb_obj : 5 nb_hashtags : 4 time to prepare the origin masks : 0.6261770725250244 time for calcul the mask position with numpy : 0.02140498161315918 nb_pixel_total : 1208227 time to create 1 rle with new method : 0.05247354507446289 time for calcul the mask position with numpy : 0.006907463073730469 nb_pixel_total : 229 time to create 1 rle with old method : 0.00046563148498535156 time for calcul the mask position with numpy : 0.006636619567871094 nb_pixel_total : 5078 time to create 1 rle with old method : 0.008004188537597656 time for calcul the mask position with numpy : 0.023850679397583008 nb_pixel_total : 767443 time to create 1 rle with new method : 0.04457283020019531 time for calcul the mask position with numpy : 0.006920337677001953 nb_pixel_total : 90485 time to create 1 rle with old method : 0.09869790077209473 time for calcul the mask position with numpy : 0.007151603698730469 nb_pixel_total : 2138 time to create 1 rle with old method : 0.002532482147216797 create new chi : 0.28942251205444336 time to delete rle : 0.0005850791931152344 batch 1 Loaded 6 chid ids of type : 3726 Number RLEs to save : 5244 TO DO : save crop sub photo not yet done ! save time : 0.459181547164917 nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 0.3225440979003906 time for calcul the mask position with numpy : 0.0165250301361084 nb_pixel_total : 1615838 time to create 1 rle with new method : 0.03371405601501465 time for calcul the mask position with numpy : 0.006644725799560547 nb_pixel_total : 99525 time to create 1 rle with old method : 0.10670208930969238 time for calcul the mask position with numpy : 0.0069370269775390625 nb_pixel_total : 16090 time to create 1 rle with old method : 0.017193078994750977 time for calcul the mask position with numpy : 0.007922887802124023 nb_pixel_total : 327883 time to create 1 rle with new method : 0.032061100006103516 time for calcul the mask position with numpy : 0.00615692138671875 nb_pixel_total : 14264 time to create 1 rle with old method : 0.01592540740966797 create new chi : 0.2517569065093994 time to delete rle : 0.0004775524139404297 batch 1 Loaded 5 chid ids of type : 3726 Number RLEs to save : 5116 TO DO : save crop sub photo not yet done ! save time : 0.3340637683868408 nb_obj : 2 nb_hashtags : 1 time to prepare the origin masks : 0.17842888832092285 time for calcul the mask position with numpy : 0.02167224884033203 nb_pixel_total : 1701928 time to create 1 rle with new method : 0.03752946853637695 time for calcul the mask position with numpy : 0.008058309555053711 nb_pixel_total : 370362 time to create 1 rle with new method : 0.028585433959960938 time for calcul the mask position with numpy : 0.006184101104736328 nb_pixel_total : 1310 time to create 1 rle with old method : 0.0015099048614501953 create new chi : 0.1047365665435791 time to delete rle : 0.00040149688720703125 batch 1 Loaded 3 chid ids of type : 3726 Number RLEs to save : 4414 TO DO : save crop sub photo not yet done ! save time : 0.7707886695861816 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.33468031883239746 time for calcul the mask position with numpy : 0.01924729347229004 nb_pixel_total : 1049083 time to create 1 rle with new method : 0.04443550109863281 time for calcul the mask position with numpy : 0.006918191909790039 nb_pixel_total : 61902 time to create 1 rle with old method : 0.06371474266052246 time for calcul the mask position with numpy : 0.007171630859375 nb_pixel_total : 106498 time to create 1 rle with old method : 0.10800671577453613 time for calcul the mask position with numpy : 0.015094757080078125 nb_pixel_total : 856117 time to create 1 rle with new method : 0.04385948181152344 create new chi : 0.31271839141845703 time to delete rle : 0.0005929470062255859 batch 1 Loaded 4 chid ids of type : 3726 Number RLEs to save : 5556 TO DO : save crop sub photo not yet done ! save time : 0.4477250576019287 nb_obj : 4 nb_hashtags : 1 time to prepare the origin masks : 0.3538498878479004 time for calcul the mask position with numpy : 0.024257898330688477 nb_pixel_total : 1734955 time to create 1 rle with new method : 0.04406857490539551 time for calcul the mask position with numpy : 0.00668644905090332 nb_pixel_total : 7349 time to create 1 rle with old method : 0.008204460144042969 time for calcul the mask position with numpy : 0.007740020751953125 nb_pixel_total : 262578 time to create 1 rle with new method : 0.04128694534301758 time for calcul the mask position with numpy : 0.006757974624633789 nb_pixel_total : 62554 time to create 1 rle with old method : 0.0646820068359375 time for calcul the mask position with numpy : 0.007062196731567383 nb_pixel_total : 6164 time to create 1 rle with old method : 0.006362199783325195 create new chi : 0.2256467342376709 time to delete rle : 0.0005278587341308594 batch 1 Loaded 5 chid ids of type : 3726 Number RLEs to save : 3713 TO DO : save crop sub photo not yet done ! save time : 0.23555874824523926 nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 0.3672952651977539 time for calcul the mask position with numpy : 0.02058100700378418 nb_pixel_total : 994514 time to create 1 rle with new method : 0.05240607261657715 time for calcul the mask position with numpy : 0.006666660308837891 nb_pixel_total : 9370 time to create 1 rle with old method : 0.009595632553100586 time for calcul the mask position with numpy : 0.006465911865234375 nb_pixel_total : 1609 time to create 1 rle with old method : 0.0017480850219726562 time for calcul the mask position with numpy : 0.007032632827758789 nb_pixel_total : 117071 time to create 1 rle with old method : 0.12279701232910156 time for calcul the mask position with numpy : 0.016239166259765625 nb_pixel_total : 951036 time to create 1 rle with new method : 0.05166196823120117 create new chi : 0.30680155754089355 time to delete rle : 0.0009114742279052734 batch 1 Loaded 5 chid ids of type : 3726 Number RLEs to save : 6478 TO DO : save crop sub photo not yet done ! save time : 0.348660945892334 nb_obj : 3 nb_hashtags : 1 time to prepare the origin masks : 0.3579063415527344 time for calcul the mask position with numpy : 0.01658153533935547 nb_pixel_total : 1035209 time to create 1 rle with new method : 0.051958560943603516 time for calcul the mask position with numpy : 0.008071184158325195 nb_pixel_total : 172836 time to create 1 rle with new method : 0.048944711685180664 time for calcul the mask position with numpy : 0.00669550895690918 nb_pixel_total : 36208 time to create 1 rle with old method : 0.03705096244812012 time for calcul the mask position with numpy : 0.01661086082458496 nb_pixel_total : 829347 time to create 1 rle with new method : 0.04644942283630371 create new chi : 0.24937987327575684 time to delete rle : 0.0006051063537597656 batch 1 Loaded 4 chid ids of type : 3726 Number RLEs to save : 5106 TO DO : save crop sub photo not yet done ! save time : 0.45062994956970215 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.3580434322357178 time for calcul the mask position with numpy : 0.020738601684570312 nb_pixel_total : 1463773 time to create 1 rle with new method : 0.045586347579956055 time for calcul the mask position with numpy : 0.011578083038330078 nb_pixel_total : 507275 time to create 1 rle with new method : 0.04004049301147461 time for calcul the mask position with numpy : 0.006866455078125 nb_pixel_total : 60702 time to create 1 rle with old method : 0.08210992813110352 time for calcul the mask position with numpy : 0.007153034210205078 nb_pixel_total : 41850 time to create 1 rle with old method : 0.06783652305603027 create new chi : 0.28715085983276367 time to delete rle : 0.0007212162017822266 batch 1 Loaded 4 chid ids of type : 3726 Number RLEs to save : 4749 TO DO : save crop sub photo not yet done ! save time : 0.33570241928100586 nb_obj : 5 nb_hashtags : 3 time to prepare the origin masks : 0.4517529010772705 time for calcul the mask position with numpy : 0.020444631576538086 nb_pixel_total : 1272706 time to create 1 rle with new method : 0.04017901420593262 time for calcul the mask position with numpy : 0.006483793258666992 nb_pixel_total : 3774 time to create 1 rle with old method : 0.0045163631439208984 time for calcul the mask position with numpy : 0.006562709808349609 nb_pixel_total : 454 time to create 1 rle with old method : 0.0007812976837158203 time for calcul the mask position with numpy : 0.006547689437866211 nb_pixel_total : 35021 time to create 1 rle with old method : 0.041790008544921875 time for calcul the mask position with numpy : 0.00688481330871582 nb_pixel_total : 29278 time to create 1 rle with old method : 0.035268306732177734 time for calcul the mask position with numpy : 0.01184988021850586 nb_pixel_total : 732367 time to create 1 rle with new method : 0.032965898513793945 create new chi : 0.2174825668334961 time to delete rle : 0.0005021095275878906 batch 1 Loaded 6 chid ids of type : 3726 Number RLEs to save : 4808 TO DO : save crop sub photo not yet done ! save time : 0.29847121238708496 map_output_result : {1333154110: (0.0, 'Should be the crop_list due to order', 0.0), 1333154091: (0.0, 'Should be the crop_list due to order', 0.0), 1333154077: (0.0, 'Should be the crop_list due to order', 0.0), 1333153911: (0.0, 'Should be the crop_list due to order', 0.0), 1333153906: (0.0, 'Should be the crop_list due to order', 0.0), 1333153900: (0.0, 'Should be the crop_list due to order', 0.0), 1333153895: (0.0, 'Should be the crop_list due to order', 0.0), 1333153887: (0.0, 'Should be the crop_list due to order', 0.0), 1333153884: (0.0, 'Should be the crop_list due to order', 0.0), 1333153878: (0.0, 'Should be the crop_list due to order', 0.0), 1333153875: (0.0, 'Should be the crop_list due to order', 0.0), 1333153873: (0.0, 'Should be the crop_list due to order', 0.0), 1333153869: (0.0, 'Should be the crop_list due to order', 0.0), 1333153864: (0.0, 'Should be the crop_list due to order', 0.0), 1333153858: (0.0, 'Should be the crop_list due to order', 0.0), 1333153729: (0.0, 'Should be the crop_list due to order', 0.0), 1333153728: (0.0, 'Should be the crop_list due to order', 0.0), 1333153723: (0.0, 'Should be the crop_list due to order', 0.0), 1333153718: (0.0, 'Should be the crop_list due to order', 0.0), 1333153713: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1333154110, 1333154091, 1333154077, 1333153911, 1333153906, 1333153900, 1333153895, 1333153887, 1333153884, 1333153878, 1333153875, 1333153873, 1333153869, 1333153864, 1333153858, 1333153729, 1333153728, 1333153723, 1333153718, 1333153713] Looping around the photos to save general results len do output : 20 /1333154110.Didn't retrieve data . /1333154091.Didn't retrieve data . /1333154077.Didn't retrieve data . /1333153911.Didn't retrieve data . /1333153906.Didn't retrieve data . /1333153900.Didn't retrieve data . /1333153895.Didn't retrieve data . /1333153887.Didn't retrieve data . /1333153884.Didn't retrieve data . /1333153878.Didn't retrieve data . /1333153875.Didn't retrieve data . /1333153873.Didn't retrieve data . /1333153869.Didn't retrieve data . /1333153864.Didn't retrieve data . /1333153858.Didn't retrieve data . /1333153729.Didn't retrieve data . /1333153728.Didn't retrieve data . /1333153723.Didn't retrieve data . /1333153718.Didn't retrieve data . /1333153713.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333154110', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333154091', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333154077', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153911', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153906', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153900', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153895', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153887', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153884', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153878', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153875', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153873', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153869', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153864', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153858', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153729', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153728', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153723', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153718', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153713', None, None, None, None, None, '2533239') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 60 time used for this insertion : 0.01477670669555664 save_final save missing photos in datou_result : time spend for datou_step_exec : 24.22654962539673 time spend to save output : 0.015834808349609375 total time spend for step 5 : 24.242384433746338 step6:crop_condition Wed Feb 5 11:27:26 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 20 ! batch 1 Loaded 115 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738751247_3455466 we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.1181154251098633 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738751249_3455466 we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.9723212718963623 we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 4869462 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738751251_3455466 we have uploaded 2 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.3980543613433838 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 4869462 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738751254_3455466 we have uploaded 5 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.288496732711792 we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1333154110, 1333154091, 1333154077, 1333153911, 1333153906, 1333153900, 1333153895, 1333153887, 1333153884, 1333153878, 1333153875, 1333153873, 1333153869, 1333153864, 1333153858, 1333153729, 1333153728, 1333153723, 1333153718, 1333153713] Looping around the photos to save general results len do output : 13 /1334896347Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334896348Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334896349Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334896351Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334896352Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334896353Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334896354Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334896355Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334896357Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334896358Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334896359Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334896360Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334896361Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333154110', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333154091', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333154077', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153911', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153906', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153900', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153895', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153887', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153884', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153878', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153875', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153873', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153869', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153864', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153858', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153729', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153728', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153723', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153718', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153713', None, None, None, None, None, '2533239') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 59 time used for this insertion : 0.8891313076019287 save_final save missing photos in datou_result : time spend for datou_step_exec : 10.303989171981812 time spend to save output : 0.8897829055786133 total time spend for step 6 : 11.193772077560425 step7:ventilate_hashtags_in_portfolio Wed Feb 5 11:27:37 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 20105213 get user id for portfolio 20105213 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20105213 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('autre_refus','Teint_Dans_La_Masse','metal','flou','Carton_brun','kraft','papier','plastique','cartonnette','environnement','mal_croppe','Carton_gris')) AND mptpi.`min_score`=0.5 To do To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20105213 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('autre_refus','Teint_Dans_La_Masse','metal','flou','Carton_brun','kraft','papier','plastique','cartonnette','environnement','mal_croppe','Carton_gris')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/20245224,20245226,20245227,20245228,20245232,20245242,20245250,20245256,20245258,20245263,20245265,20245270?tags=Carton_gris,papier,flou,Teint_Dans_La_Masse,environnement,mal_croppe,Carton_brun,metal,plastique,cartonnette,autre_refus,kraft Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1333154110, 1333154091, 1333154077, 1333153911, 1333153906, 1333153900, 1333153895, 1333153887, 1333153884, 1333153878, 1333153875, 1333153873, 1333153869, 1333153864, 1333153858, 1333153729, 1333153728, 1333153723, 1333153718, 1333153713] Looping around the photos to save general results len do output : 1 /20105213. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333154110', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333154091', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333154077', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153911', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153906', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153900', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153895', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153887', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153884', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153878', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153875', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153873', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153869', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153864', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153858', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153729', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153728', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153723', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153718', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153713', None, None, None, None, None, '2533239') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 21 time used for this insertion : 0.015672683715820312 save_final save missing photos in datou_result : time spend for datou_step_exec : 3.0366508960723877 time spend to save output : 0.015994787216186523 total time spend for step 7 : 3.052645683288574 step8:final Wed Feb 5 11:27:40 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1333154110: ('0.022770331777479173',), 1333154091: ('0.022770331777479173',), 1333154077: ('0.022770331777479173',), 1333153911: ('0.022770331777479173',), 1333153906: ('0.022770331777479173',), 1333153900: ('0.022770331777479173',), 1333153895: ('0.022770331777479173',), 1333153887: ('0.022770331777479173',), 1333153884: ('0.022770331777479173',), 1333153878: ('0.022770331777479173',), 1333153875: ('0.022770331777479173',), 1333153873: ('0.022770331777479173',), 1333153869: ('0.022770331777479173',), 1333153864: ('0.022770331777479173',), 1333153858: ('0.022770331777479173',), 1333153729: ('0.022770331777479173',), 1333153728: ('0.022770331777479173',), 1333153723: ('0.022770331777479173',), 1333153718: ('0.022770331777479173',), 1333153713: ('0.022770331777479173',)} new output for save of step final : {1333154110: ('0.022770331777479173',), 1333154091: ('0.022770331777479173',), 1333154077: ('0.022770331777479173',), 1333153911: ('0.022770331777479173',), 1333153906: ('0.022770331777479173',), 1333153900: ('0.022770331777479173',), 1333153895: ('0.022770331777479173',), 1333153887: ('0.022770331777479173',), 1333153884: ('0.022770331777479173',), 1333153878: ('0.022770331777479173',), 1333153875: ('0.022770331777479173',), 1333153873: ('0.022770331777479173',), 1333153869: ('0.022770331777479173',), 1333153864: ('0.022770331777479173',), 1333153858: ('0.022770331777479173',), 1333153729: ('0.022770331777479173',), 1333153728: ('0.022770331777479173',), 1333153723: ('0.022770331777479173',), 1333153718: ('0.022770331777479173',), 1333153713: ('0.022770331777479173',)} [1333154110, 1333154091, 1333154077, 1333153911, 1333153906, 1333153900, 1333153895, 1333153887, 1333153884, 1333153878, 1333153875, 1333153873, 1333153869, 1333153864, 1333153858, 1333153729, 1333153728, 1333153723, 1333153718, 1333153713] Looping around the photos to save general results len do output : 20 /1333154110.Didn't retrieve data . /1333154091.Didn't retrieve data . /1333154077.Didn't retrieve data . /1333153911.Didn't retrieve data . /1333153906.Didn't retrieve data . /1333153900.Didn't retrieve data . /1333153895.Didn't retrieve data . /1333153887.Didn't retrieve data . /1333153884.Didn't retrieve data . /1333153878.Didn't retrieve data . /1333153875.Didn't retrieve data . /1333153873.Didn't retrieve data . /1333153869.Didn't retrieve data . /1333153864.Didn't retrieve data . /1333153858.Didn't retrieve data . /1333153729.Didn't retrieve data . /1333153728.Didn't retrieve data . /1333153723.Didn't retrieve data . /1333153718.Didn't retrieve data . /1333153713.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333154110', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333154091', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333154077', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153911', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153906', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153900', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153895', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153887', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153884', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153878', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153875', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153873', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153869', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153864', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153858', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153729', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153728', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153723', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153718', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153713', None, None, None, None, None, '2533239') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 60 time used for this insertion : 0.014004707336425781 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.8467707633972168 time spend to save output : 0.014885187149047852 total time spend for step 8 : 1.8616559505462646 step9:velours_tree Wed Feb 5 11:27:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.036559343338012695 time spend to save output : 3.123283386230469e-05 total time spend for step 9 : 0.036590576171875 step10:send_mail_cod Wed Feb 5 11:27:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin in order to get the selector url, please entre the license of selector results_Auto_P20105213_05-02-2025_11_27_42.pdf 20245227 imagette202452271738751262 20245242 imagette202452421738751262 20245256 change filename to text .imagette202452561738751262 20245258 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette202452581738751262 20245263 change filename to text .change filename to text .imagette202452631738751264 20245265 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette202452651738751265 20245270 imagette202452701738751265 20245224 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette202452241738751265 20245228 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette202452281738751266 20245250 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette202452501738751268 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20105213 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20245224,20245226,20245227,20245228,20245232,20245242,20245250,20245256,20245258,20245263,20245265,20245270?tags=Carton_gris,papier,flou,Teint_Dans_La_Masse,environnement,mal_croppe,Carton_brun,metal,plastique,cartonnette,autre_refus,kraft your option no_mail is active, we will not send the real mail to your client args[1333154110] : ('0.022770331777479173',) no score found for photo 1333154110 We are sending mail with results at report@fotonower.com args[1333154091] : ('0.022770331777479173',) no score found for photo 1333154091 We are sending mail with results at report@fotonower.com args[1333154077] : ('0.022770331777479173',) no score found for photo 1333154077 We are sending mail with results at report@fotonower.com args[1333153911] : ('0.022770331777479173',) no score found for photo 1333153911 We are sending mail with results at report@fotonower.com args[1333153906] : ('0.022770331777479173',) no score found for photo 1333153906 We are sending mail with results at report@fotonower.com args[1333153900] : ('0.022770331777479173',) no score found for photo 1333153900 We are sending mail with results at report@fotonower.com args[1333153895] : ('0.022770331777479173',) no score found for photo 1333153895 We are sending mail with results at report@fotonower.com args[1333153887] : ('0.022770331777479173',) no score found for photo 1333153887 We are sending mail with results at report@fotonower.com args[1333153884] : ('0.022770331777479173',) no score found for photo 1333153884 We are sending mail with results at report@fotonower.com args[1333153878] : ('0.022770331777479173',) no score found for photo 1333153878 We are sending mail with results at report@fotonower.com args[1333153875] : ('0.022770331777479173',) no score found for photo 1333153875 We are sending mail with results at report@fotonower.com args[1333153873] : ('0.022770331777479173',) no score found for photo 1333153873 We are sending mail with results at report@fotonower.com args[1333153869] : ('0.022770331777479173',) no score found for photo 1333153869 We are sending mail with results at report@fotonower.com args[1333153864] : ('0.022770331777479173',) no score found for photo 1333153864 We are sending mail with results at report@fotonower.com args[1333153858] : ('0.022770331777479173',) no score found for photo 1333153858 We are sending mail with results at report@fotonower.com args[1333153729] : ('0.022770331777479173',) no score found for photo 1333153729 We are sending mail with results at report@fotonower.com args[1333153728] : ('0.022770331777479173',) no score found for photo 1333153728 We are sending mail with results at report@fotonower.com args[1333153723] : ('0.022770331777479173',) no score found for photo 1333153723 We are sending mail with results at report@fotonower.com args[1333153718] : ('0.022770331777479173',) no score found for photo 1333153718 We are sending mail with results at report@fotonower.com args[1333153713] : ('0.022770331777479173',) no score found for photo 1333153713 We are sending mail with results at report@fotonower.com refus_total : 0.022770331777479173 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20105213 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1333120751,1333120936,1333149979,1333121070,1333150950,1333149875,1333149959,1333149962,1333149966,1333149971,1333149974,1333149976,1333149981,1333149983,1333150497,1333150668,1333150822,1333150914,1333150946,1333152384) Found this number of photos: 20 begin to download photo : 1333120751 begin to download photo : 1333149875 begin to download photo : 1333149974 begin to download photo : 1333150668 download finish for photo 1333120751 begin to download photo : 1333120936 download finish for photo 1333149974 begin to download photo : 1333149976 download finish for photo 1333150668 begin to download photo : 1333150822 download finish for photo 1333149875 begin to download photo : 1333149959 download finish for photo 1333120936 begin to download photo : 1333149979 download finish for photo 1333150822 begin to download photo : 1333150914 download finish for photo 1333149976 begin to download photo : 1333149981 download finish for photo 1333149959 begin to download photo : 1333149962 download finish for photo 1333149979 begin to download photo : 1333121070 download finish for photo 1333149962 begin to download photo : 1333149966 download finish for photo 1333150914 begin to download photo : 1333150946 download finish for photo 1333149981 begin to download photo : 1333149983 download finish for photo 1333121070 begin to download photo : 1333150950 download finish for photo 1333150946 begin to download photo : 1333152384 download finish for photo 1333149983 begin to download photo : 1333150497 download finish for photo 1333149966 begin to download photo : 1333149971 download finish for photo 1333152384 download finish for photo 1333150497 download finish for photo 1333149971 download finish for photo 1333150950 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20105213_05-02-2025_11_27_42.pdf results_Auto_P20105213_05-02-2025_11_27_42.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20105213_05-02-2025_11_27_42.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3459','20105213','results_Auto_P20105213_05-02-2025_11_27_42.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20105213_05-02-2025_11_27_42.pdf','pdf','','0.44','0.022770331777479173') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1333154110, 1333154091, 1333154077, 1333153911, 1333153906, 1333153900, 1333153895, 1333153887, 1333153884, 1333153878, 1333153875, 1333153873, 1333153869, 1333153864, 1333153858, 1333153729, 1333153728, 1333153723, 1333153718, 1333153713] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333154110', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333154091', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333154077', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153911', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153906', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153900', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153895', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153887', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153884', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153878', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153875', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153873', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153869', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153864', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153858', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153729', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153728', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153723', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153718', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153713', None, None, None, None, None, '2533239') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 20 time used for this insertion : 0.017398595809936523 save_final save missing photos in datou_result : time spend for datou_step_exec : 11.97071123123169 time spend to save output : 0.01773667335510254 total time spend for step 10 : 11.988447904586792 step11:split_time_score Wed Feb 5 11:27:54 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('23', 264),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 30012025 20105213 Nombre de photos uploadées : 264 / 23040 (1%) 30012025 20105213 Nombre de photos taguées (types de déchets): 0 / 264 (0%) 30012025 20105213 Nombre de photos taguées (volume) : 0 / 264 (0%) elapsed_time : load_data_split_time_score 3.0994415283203125e-06 elapsed_time : order_list_meta_photo_and_scores 8.58306884765625e-06 ???????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.009696245193481445 elapsed_time : insert_dashboard_record_day_entry 0.024190425872802734 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.03875924830956473 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20073646_30-01-2025_14_22_06.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20073646 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20073646 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20073158 order by id desc limit 1 Qualite : 0.032095205732650345 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20073159_30-01-2025_13_02_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20073159 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20073159 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20072921 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20072383 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20072384 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20072385 order by id desc limit 1 Qualite : 0.046647691037909354 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20068238_30-01-2025_09_43_56.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20068238 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20068238 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20072386 order by id desc limit 1 Qualite : 0.033926560641630986 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20072387_30-01-2025_12_32_52.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20072387 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20072387 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20071969 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20071970 order by id desc limit 1 Qualite : 0.027665320859549338 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20071972_30-01-2025_12_02_45.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20071972 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20071972 AND mptpi.`type`=3726 To do Qualite : 0.0061049359209256315 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20138070_02-02-2025_04_30_52.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20138070 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20138070 AND mptpi.`type`=3726 To do Qualite : 0.014301945230445801 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20138071_02-02-2025_02_40_52.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20138071 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20138071 AND mptpi.`type`=3726 To do Qualite : 0.017561565375200358 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20138072_02-02-2025_03_51_01.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20138072 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20138072 AND mptpi.`type`=3726 To do Qualite : 0.027302961031116697 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20138073_02-02-2025_01_51_05.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20138073 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20138073 AND mptpi.`type`=3726 To do Qualite : 0.030643068324198868 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20138074_02-02-2025_01_00_53.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20138074 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20138074 AND mptpi.`type`=3726 To do Qualite : 0.02387720144099549 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20138075_01-02-2025_23_40_33.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20138075 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20138075 AND mptpi.`type`=3726 To do Qualite : 0.034037250830284284 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20138076_01-02-2025_20_51_18.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20138076 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20138076 AND mptpi.`type`=3726 To do Qualite : 0.01574846168747699 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20138077_01-02-2025_22_20_36.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20138077 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20138077 AND mptpi.`type`=3726 To do Qualite : 0.03791186739045397 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20138078_01-02-2025_20_00_45.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20138078 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20138078 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20105154 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20105155 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20105156 order by id desc limit 1 Qualite : 0.030667975954369344 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20081910_04-02-2025_11_15_33.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20081910 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20081910 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20105157 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20105158 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20105159 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20105160 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20105161 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20105162 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20105163 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20105164 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20105165 order by id desc limit 1 Qualite : 0.006845739043700519 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20105166_05-02-2025_08_54_57.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20105166 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20105166 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20105167 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20105168 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20105193 order by id desc limit 1 Qualite : 0.013426174135866055 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20088595_30-01-2025_23_11_28.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20088595 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20088595 AND mptpi.`type`=3726 To do Qualite : 0.011734877380199571 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20088399_04-02-2025_11_14_47.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20088399 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20088399 AND mptpi.`type`=3726 To do Qualite : 0.01605529895523676 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20105211_05-02-2025_10_31_13.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20105211 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20105211 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20105212 order by id desc limit 1 Qualite : 0.022770331777479173 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20105213_05-02-2025_11_27_42.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20105213 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20105213 AND mptpi.`type`=3726 To do Qualite : 0.024420791162820855 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20105214_05-02-2025_07_55_27.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20105214 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20105214 AND mptpi.`type`=3726 To do Qualite : 0.037600121819392304 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20105215_05-02-2025_10_34_31.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20105215 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20105215 AND mptpi.`type`=3726 To do Qualite : 0.024457677482647447 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20105216_05-02-2025_06_19_29.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20105216 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20105216 AND mptpi.`type`=3726 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'30012025': {'nb_upload': 264, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1333154110, 1333154091, 1333154077, 1333153911, 1333153906, 1333153900, 1333153895, 1333153887, 1333153884, 1333153878, 1333153875, 1333153873, 1333153869, 1333153864, 1333153858, 1333153729, 1333153728, 1333153723, 1333153718, 1333153713] Looping around the photos to save general results len do output : 1 /20105213Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333154110', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333154091', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333154077', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153911', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153906', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153900', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153895', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153887', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153884', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153878', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153875', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153873', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153869', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153864', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153858', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153729', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153728', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153723', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153718', None, None, None, None, None, '2533239') ('3459', None, None, None, None, None, None, None, '2533239') ('3459', '20105213', '1333153713', None, None, None, None, None, '2533239') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 21 time used for this insertion : 0.12441301345825195 save_final save missing photos in datou_result : time spend for datou_step_exec : 25.790571689605713 time spend to save output : 0.12476277351379395 total time spend for step 11 : 25.915334463119507 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 58.86user 23.15system 2:37.99elapsed 51%CPU (0avgtext+0avgdata 2978300maxresident)k 11104inputs+52208outputs (7major+2052848minor)pagefaults 0swaps