python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 3459 -l 10 ' -s datou_current_3459 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 336713 load datou : 3459 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) load thcls load THCL from format json or kwargs add thcl : 2456 in CacheModelConfig load THCL from format json or kwargs add thcl : 2868 in CacheModelConfig load THCL from format json or kwargs add thcl : 2896 in CacheModelConfig load pdts add pdt : 4999 in CacheModelConfig add pdt : 5288 in CacheModelConfig add pdt : 5309 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 we have a portfolio with more photos than limit : 22>10 please execute split_portfolio.py -i 20699439 -l 10 size over we load limit photo not treated list_input_json: [] Current got : datou_id : 3459, datou_cur_ids : ['2601855'] with mtr_portfolio_ids : ['20699439'] and first list_photo_ids : [] new path : /proc/336713/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, thcl, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 10 ; length of list_pids : 10 ; length of list_args : 10 time to download the photos : 2.124056100845337 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Fri Feb 21 01:46:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 5300 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-21 01:46:31.950436: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-21 01:46:31.975155: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-21 01:46:31.977236: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f7544000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-21 01:46:31.977301: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-21 01:46:31.981475: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-21 01:46:32.143384: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x3b7caea0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-21 01:46:32.143447: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-21 01:46:32.144725: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-21 01:46:32.145165: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-21 01:46:32.148509: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-21 01:46:32.192840: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-21 01:46:32.193380: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-21 01:46:32.197773: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-21 01:46:32.199428: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-21 01:46:32.207263: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-21 01:46:32.208481: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-21 01:46:32.208570: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-21 01:46:32.209174: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-21 01:46:32.209190: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-21 01:46:32.209199: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-21 01:46:32.210185: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4839 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-21 01:46:32.499553: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-21 01:46:32.499712: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-21 01:46:32.499738: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-21 01:46:32.499761: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-21 01:46:32.499782: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-21 01:46:32.499803: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-21 01:46:32.499824: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-21 01:46:32.499845: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-21 01:46:32.500980: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-21 01:46:32.502303: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-21 01:46:32.502384: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-21 01:46:32.502406: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-21 01:46:32.502426: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-21 01:46:32.502446: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-21 01:46:32.502465: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-21 01:46:32.502484: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-21 01:46:32.502503: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-21 01:46:32.503653: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-21 01:46:32.503702: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-21 01:46:32.503713: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-21 01:46:32.503722: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-21 01:46:32.504721: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4839 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-21 01:46:42.839355: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-21 01:46:43.064185: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 10 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 21 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 14 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 20 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.01745 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 15 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 20 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 17 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 22 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 12 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 16 Detection mask done ! Trying to reset tf kernel 337240 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 11 tf kernel not reseted sub process len(results) : 10 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 10 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5300 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.004557132720947266 nb_pixel_total : 125236 time to create 1 rle with old method : 0.1422407627105713 length of segment : 560 time for calcul the mask position with numpy : 0.0034656524658203125 nb_pixel_total : 108633 time to create 1 rle with old method : 0.1264188289642334 length of segment : 321 time for calcul the mask position with numpy : 0.0006604194641113281 nb_pixel_total : 25080 time to create 1 rle with old method : 0.02927565574645996 length of segment : 182 time for calcul the mask position with numpy : 0.0007328987121582031 nb_pixel_total : 21585 time to create 1 rle with old method : 0.024248361587524414 length of segment : 141 time for calcul the mask position with numpy : 0.0023491382598876953 nb_pixel_total : 73516 time to create 1 rle with old method : 0.08154177665710449 length of segment : 375 time for calcul the mask position with numpy : 0.00015997886657714844 nb_pixel_total : 1681 time to create 1 rle with old method : 0.001912832260131836 length of segment : 72 time for calcul the mask position with numpy : 0.04106903076171875 nb_pixel_total : 1543112 time to create 1 rle with new method : 0.34574413299560547 length of segment : 2234 time for calcul the mask position with numpy : 0.0016655921936035156 nb_pixel_total : 80747 time to create 1 rle with old method : 0.09222793579101562 length of segment : 459 time for calcul the mask position with numpy : 0.0006930828094482422 nb_pixel_total : 19102 time to create 1 rle with old method : 0.022819042205810547 length of segment : 215 time for calcul the mask position with numpy : 0.00025653839111328125 nb_pixel_total : 6652 time to create 1 rle with old method : 0.008018016815185547 length of segment : 98 time for calcul the mask position with numpy : 0.0018641948699951172 nb_pixel_total : 73456 time to create 1 rle with old method : 0.0855855941772461 length of segment : 369 time for calcul the mask position with numpy : 0.0015888214111328125 nb_pixel_total : 70654 time to create 1 rle with old method : 0.0819849967956543 length of segment : 317 time for calcul the mask position with numpy : 0.0015804767608642578 nb_pixel_total : 86959 time to create 1 rle with old method : 0.10030293464660645 length of segment : 416 time for calcul the mask position with numpy : 0.0012013912200927734 nb_pixel_total : 35058 time to create 1 rle with old method : 0.04208827018737793 length of segment : 293 time for calcul the mask position with numpy : 0.0015017986297607422 nb_pixel_total : 39156 time to create 1 rle with old method : 0.04626870155334473 length of segment : 362 time for calcul the mask position with numpy : 0.003100872039794922 nb_pixel_total : 153807 time to create 1 rle with new method : 0.005582332611083984 length of segment : 393 time for calcul the mask position with numpy : 0.007338762283325195 nb_pixel_total : 189035 time to create 1 rle with new method : 0.012138128280639648 length of segment : 635 time for calcul the mask position with numpy : 0.0010669231414794922 nb_pixel_total : 14168 time to create 1 rle with old method : 0.016666650772094727 length of segment : 232 time for calcul the mask position with numpy : 0.0031194686889648438 nb_pixel_total : 98475 time to create 1 rle with old method : 0.13539648056030273 length of segment : 450 time for calcul the mask position with numpy : 0.00019621849060058594 nb_pixel_total : 2920 time to create 1 rle with old method : 0.003701925277709961 length of segment : 84 time for calcul the mask position with numpy : 0.0008494853973388672 nb_pixel_total : 22930 time to create 1 rle with old method : 0.0272982120513916 length of segment : 324 time for calcul the mask position with numpy : 0.0008580684661865234 nb_pixel_total : 26527 time to create 1 rle with old method : 0.030780553817749023 length of segment : 258 time for calcul the mask position with numpy : 0.0026967525482177734 nb_pixel_total : 129659 time to create 1 rle with old method : 0.14898252487182617 length of segment : 446 time for calcul the mask position with numpy : 0.0019528865814208984 nb_pixel_total : 97658 time to create 1 rle with old method : 0.11562967300415039 length of segment : 457 time for calcul the mask position with numpy : 0.001806020736694336 nb_pixel_total : 80657 time to create 1 rle with old method : 0.09086418151855469 length of segment : 310 time for calcul the mask position with numpy : 0.0002048015594482422 nb_pixel_total : 2199 time to create 1 rle with old method : 0.0027663707733154297 length of segment : 75 time for calcul the mask position with numpy : 0.00023508071899414062 nb_pixel_total : 3710 time to create 1 rle with old method : 0.004728078842163086 length of segment : 91 time for calcul the mask position with numpy : 0.009255647659301758 nb_pixel_total : 380934 time to create 1 rle with new method : 0.01911783218383789 length of segment : 795 time for calcul the mask position with numpy : 0.0002541542053222656 nb_pixel_total : 2384 time to create 1 rle with old method : 0.0031425952911376953 length of segment : 82 time for calcul the mask position with numpy : 0.0009579658508300781 nb_pixel_total : 41551 time to create 1 rle with old method : 0.04924774169921875 length of segment : 274 time for calcul the mask position with numpy : 9.036064147949219e-05 nb_pixel_total : 1411 time to create 1 rle with old method : 0.0019147396087646484 length of segment : 46 time for calcul the mask position with numpy : 0.00015401840209960938 nb_pixel_total : 1546 time to create 1 rle with old method : 0.0019769668579101562 length of segment : 73 time for calcul the mask position with numpy : 0.0029327869415283203 nb_pixel_total : 96288 time to create 1 rle with old method : 0.10811591148376465 length of segment : 423 time for calcul the mask position with numpy : 0.00035381317138671875 nb_pixel_total : 5184 time to create 1 rle with old method : 0.006175994873046875 length of segment : 142 time for calcul the mask position with numpy : 0.002626180648803711 nb_pixel_total : 104009 time to create 1 rle with old method : 0.11456489562988281 length of segment : 361 time for calcul the mask position with numpy : 0.0077381134033203125 nb_pixel_total : 278560 time to create 1 rle with new method : 0.01192331314086914 length of segment : 731 time for calcul the mask position with numpy : 0.0020132064819335938 nb_pixel_total : 97572 time to create 1 rle with old method : 0.1125798225402832 length of segment : 459 time for calcul the mask position with numpy : 0.0005125999450683594 nb_pixel_total : 5696 time to create 1 rle with old method : 0.007028341293334961 length of segment : 103 time for calcul the mask position with numpy : 0.0016455650329589844 nb_pixel_total : 42755 time to create 1 rle with old method : 0.04921364784240723 length of segment : 421 time for calcul the mask position with numpy : 0.0009505748748779297 nb_pixel_total : 27842 time to create 1 rle with old method : 0.03262495994567871 length of segment : 264 time for calcul the mask position with numpy : 0.0021882057189941406 nb_pixel_total : 71989 time to create 1 rle with old method : 0.08344483375549316 length of segment : 311 time for calcul the mask position with numpy : 0.0009002685546875 nb_pixel_total : 20118 time to create 1 rle with old method : 0.02359771728515625 length of segment : 279 time for calcul the mask position with numpy : 0.00017976760864257812 nb_pixel_total : 2895 time to create 1 rle with old method : 0.0036361217498779297 length of segment : 78 time for calcul the mask position with numpy : 0.000293731689453125 nb_pixel_total : 3343 time to create 1 rle with old method : 0.004556894302368164 length of segment : 95 time for calcul the mask position with numpy : 0.00022077560424804688 nb_pixel_total : 6992 time to create 1 rle with old method : 0.008826494216918945 length of segment : 105 time for calcul the mask position with numpy : 0.010381937026977539 nb_pixel_total : 425686 time to create 1 rle with new method : 0.01755380630493164 length of segment : 690 time for calcul the mask position with numpy : 0.00018978118896484375 nb_pixel_total : 1577 time to create 1 rle with old method : 0.0019159317016601562 length of segment : 67 time for calcul the mask position with numpy : 0.00139617919921875 nb_pixel_total : 42139 time to create 1 rle with old method : 0.048839569091796875 length of segment : 214 time for calcul the mask position with numpy : 0.0011005401611328125 nb_pixel_total : 68600 time to create 1 rle with old method : 0.08155584335327148 length of segment : 297 time for calcul the mask position with numpy : 0.004865407943725586 nb_pixel_total : 279560 time to create 1 rle with new method : 0.011986494064331055 length of segment : 714 time for calcul the mask position with numpy : 0.0002989768981933594 nb_pixel_total : 2845 time to create 1 rle with old method : 0.0036940574645996094 length of segment : 83 time for calcul the mask position with numpy : 0.00019502639770507812 nb_pixel_total : 1451 time to create 1 rle with old method : 0.001837015151977539 length of segment : 98 time for calcul the mask position with numpy : 0.0003516674041748047 nb_pixel_total : 6624 time to create 1 rle with old method : 0.007984638214111328 length of segment : 177 time for calcul the mask position with numpy : 0.0008189678192138672 nb_pixel_total : 17335 time to create 1 rle with old method : 0.02063751220703125 length of segment : 268 time for calcul the mask position with numpy : 0.0029458999633789062 nb_pixel_total : 96504 time to create 1 rle with old method : 0.11617541313171387 length of segment : 419 time for calcul the mask position with numpy : 0.0002090930938720703 nb_pixel_total : 1954 time to create 1 rle with old method : 0.0025191307067871094 length of segment : 68 time for calcul the mask position with numpy : 0.0019154548645019531 nb_pixel_total : 75811 time to create 1 rle with old method : 0.08437800407409668 length of segment : 314 time for calcul the mask position with numpy : 0.003314971923828125 nb_pixel_total : 140202 time to create 1 rle with old method : 0.15064692497253418 length of segment : 416 time for calcul the mask position with numpy : 0.000782012939453125 nb_pixel_total : 25948 time to create 1 rle with old method : 0.029590606689453125 length of segment : 251 time for calcul the mask position with numpy : 0.0002117156982421875 nb_pixel_total : 1496 time to create 1 rle with old method : 0.0025610923767089844 length of segment : 61 time for calcul the mask position with numpy : 0.0002732276916503906 nb_pixel_total : 3256 time to create 1 rle with old method : 0.005557537078857422 length of segment : 88 time for calcul the mask position with numpy : 0.0017752647399902344 nb_pixel_total : 71233 time to create 1 rle with old method : 0.0876626968383789 length of segment : 286 time for calcul the mask position with numpy : 0.0013399124145507812 nb_pixel_total : 75853 time to create 1 rle with old method : 0.08271479606628418 length of segment : 313 time for calcul the mask position with numpy : 0.0002262592315673828 nb_pixel_total : 2115 time to create 1 rle with old method : 0.0024938583374023438 length of segment : 136 time for calcul the mask position with numpy : 0.00017786026000976562 nb_pixel_total : 2272 time to create 1 rle with old method : 0.00298309326171875 length of segment : 73 time for calcul the mask position with numpy : 0.002801656723022461 nb_pixel_total : 122120 time to create 1 rle with old method : 0.13321447372436523 length of segment : 381 time for calcul the mask position with numpy : 0.0027146339416503906 nb_pixel_total : 73397 time to create 1 rle with old method : 0.07969331741333008 length of segment : 438 time for calcul the mask position with numpy : 0.002230405807495117 nb_pixel_total : 63037 time to create 1 rle with old method : 0.07770085334777832 length of segment : 336 time for calcul the mask position with numpy : 0.0020275115966796875 nb_pixel_total : 91394 time to create 1 rle with old method : 0.09701967239379883 length of segment : 481 time for calcul the mask position with numpy : 0.00034999847412109375 nb_pixel_total : 4664 time to create 1 rle with old method : 0.005499601364135742 length of segment : 129 time for calcul the mask position with numpy : 0.00018143653869628906 nb_pixel_total : 1856 time to create 1 rle with old method : 0.0023369789123535156 length of segment : 88 time for calcul the mask position with numpy : 0.003686189651489258 nb_pixel_total : 126136 time to create 1 rle with old method : 0.13987994194030762 length of segment : 408 time for calcul the mask position with numpy : 0.0002465248107910156 nb_pixel_total : 2330 time to create 1 rle with old method : 0.0029442310333251953 length of segment : 82 time for calcul the mask position with numpy : 0.0007939338684082031 nb_pixel_total : 14998 time to create 1 rle with old method : 0.01786065101623535 length of segment : 282 time for calcul the mask position with numpy : 0.00779414176940918 nb_pixel_total : 378729 time to create 1 rle with new method : 0.016222476959228516 length of segment : 756 time for calcul the mask position with numpy : 0.00021195411682128906 nb_pixel_total : 2376 time to create 1 rle with old method : 0.0029845237731933594 length of segment : 62 time for calcul the mask position with numpy : 0.004135608673095703 nb_pixel_total : 338438 time to create 1 rle with new method : 0.01018381118774414 length of segment : 703 time for calcul the mask position with numpy : 0.003985166549682617 nb_pixel_total : 131505 time to create 1 rle with old method : 0.1425166130065918 length of segment : 482 time for calcul the mask position with numpy : 0.002934694290161133 nb_pixel_total : 129954 time to create 1 rle with old method : 0.14745187759399414 length of segment : 390 time for calcul the mask position with numpy : 8.845329284667969e-05 nb_pixel_total : 3227 time to create 1 rle with old method : 0.003834247589111328 length of segment : 86 time for calcul the mask position with numpy : 0.0007112026214599609 nb_pixel_total : 7490 time to create 1 rle with old method : 0.008913516998291016 length of segment : 161 time for calcul the mask position with numpy : 0.003168344497680664 nb_pixel_total : 110986 time to create 1 rle with old method : 0.14698076248168945 length of segment : 409 time for calcul the mask position with numpy : 0.0006282329559326172 nb_pixel_total : 16261 time to create 1 rle with old method : 0.018383264541625977 length of segment : 186 time for calcul the mask position with numpy : 0.0010578632354736328 nb_pixel_total : 33162 time to create 1 rle with old method : 0.03822898864746094 length of segment : 287 time for calcul the mask position with numpy : 0.0004379749298095703 nb_pixel_total : 19619 time to create 1 rle with old method : 0.023095369338989258 length of segment : 166 time for calcul the mask position with numpy : 0.003063201904296875 nb_pixel_total : 121574 time to create 1 rle with old method : 0.13357973098754883 length of segment : 466 time for calcul the mask position with numpy : 0.003066539764404297 nb_pixel_total : 105094 time to create 1 rle with old method : 0.12093019485473633 length of segment : 357 time for calcul the mask position with numpy : 0.00047588348388671875 nb_pixel_total : 14005 time to create 1 rle with old method : 0.016658306121826172 length of segment : 145 time for calcul the mask position with numpy : 0.00010323524475097656 nb_pixel_total : 1476 time to create 1 rle with old method : 0.0019664764404296875 length of segment : 67 time for calcul the mask position with numpy : 0.0011627674102783203 nb_pixel_total : 45161 time to create 1 rle with old method : 0.05170774459838867 length of segment : 270 time for calcul the mask position with numpy : 0.00225067138671875 nb_pixel_total : 58763 time to create 1 rle with old method : 0.06801891326904297 length of segment : 771 time for calcul the mask position with numpy : 0.0018351078033447266 nb_pixel_total : 62229 time to create 1 rle with old method : 0.07691597938537598 length of segment : 541 time for calcul the mask position with numpy : 0.0005652904510498047 nb_pixel_total : 8793 time to create 1 rle with old method : 0.010591268539428711 length of segment : 208 time for calcul the mask position with numpy : 0.0014963150024414062 nb_pixel_total : 34324 time to create 1 rle with old method : 0.04089856147766113 length of segment : 608 time for calcul the mask position with numpy : 0.0005381107330322266 nb_pixel_total : 8025 time to create 1 rle with old method : 0.009792804718017578 length of segment : 215 time for calcul the mask position with numpy : 0.0008792877197265625 nb_pixel_total : 42734 time to create 1 rle with old method : 0.11456155776977539 length of segment : 201 time for calcul the mask position with numpy : 0.0009615421295166016 nb_pixel_total : 19176 time to create 1 rle with old method : 0.023470640182495117 length of segment : 316 time for calcul the mask position with numpy : 2.019806385040283 nb_pixel_total : 4586772 time to create 1 rle with new method : 1.060279130935669 length of segment : 2269 time for calcul the mask position with numpy : 0.0007293224334716797 nb_pixel_total : 43670 time to create 1 rle with old method : 0.06923127174377441 length of segment : 181 time for calcul the mask position with numpy : 8.96453857421875e-05 nb_pixel_total : 2324 time to create 1 rle with old method : 0.0028777122497558594 length of segment : 80 time for calcul the mask position with numpy : 0.0022177696228027344 nb_pixel_total : 151010 time to create 1 rle with new method : 0.0037877559661865234 length of segment : 542 time for calcul the mask position with numpy : 0.0016126632690429688 nb_pixel_total : 52059 time to create 1 rle with old method : 0.058054447174072266 length of segment : 421 time for calcul the mask position with numpy : 0.00033211708068847656 nb_pixel_total : 8969 time to create 1 rle with old method : 0.011152982711791992 length of segment : 91 time for calcul the mask position with numpy : 0.00025081634521484375 nb_pixel_total : 5140 time to create 1 rle with old method : 0.006459236145019531 length of segment : 57 time for calcul the mask position with numpy : 0.002252340316772461 nb_pixel_total : 113358 time to create 1 rle with old method : 0.12955760955810547 length of segment : 406 time for calcul the mask position with numpy : 0.0017275810241699219 nb_pixel_total : 45974 time to create 1 rle with old method : 0.051816701889038086 length of segment : 607 time for calcul the mask position with numpy : 0.002629995346069336 nb_pixel_total : 140037 time to create 1 rle with old method : 0.15465784072875977 length of segment : 622 time for calcul the mask position with numpy : 0.0007183551788330078 nb_pixel_total : 16182 time to create 1 rle with old method : 0.01828742027282715 length of segment : 187 time for calcul the mask position with numpy : 0.0004150867462158203 nb_pixel_total : 9720 time to create 1 rle with old method : 0.011452198028564453 length of segment : 181 time for calcul the mask position with numpy : 0.001692056655883789 nb_pixel_total : 35234 time to create 1 rle with old method : 0.03888845443725586 length of segment : 300 time for calcul the mask position with numpy : 0.006273746490478516 nb_pixel_total : 271389 time to create 1 rle with new method : 0.012066125869750977 length of segment : 518 time for calcul the mask position with numpy : 0.00018835067749023438 nb_pixel_total : 6904 time to create 1 rle with old method : 0.008717060089111328 length of segment : 72 time for calcul the mask position with numpy : 0.0016033649444580078 nb_pixel_total : 49790 time to create 1 rle with old method : 0.05739784240722656 length of segment : 393 time for calcul the mask position with numpy : 0.014325618743896484 nb_pixel_total : 617713 time to create 1 rle with new method : 0.03322434425354004 length of segment : 1106 time for calcul the mask position with numpy : 0.0052373409271240234 nb_pixel_total : 147570 time to create 1 rle with old method : 0.16643691062927246 length of segment : 831 time for calcul the mask position with numpy : 0.0005772113800048828 nb_pixel_total : 14859 time to create 1 rle with old method : 0.017987728118896484 length of segment : 200 time for calcul the mask position with numpy : 0.00028967857360839844 nb_pixel_total : 13063 time to create 1 rle with old method : 0.015461921691894531 length of segment : 106 time for calcul the mask position with numpy : 0.0004336833953857422 nb_pixel_total : 11165 time to create 1 rle with old method : 0.013051271438598633 length of segment : 148 time for calcul the mask position with numpy : 0.0007319450378417969 nb_pixel_total : 23406 time to create 1 rle with old method : 0.02697277069091797 length of segment : 252 time for calcul the mask position with numpy : 0.00010538101196289062 nb_pixel_total : 880 time to create 1 rle with old method : 0.0011599063873291016 length of segment : 47 time for calcul the mask position with numpy : 0.0021173954010009766 nb_pixel_total : 77070 time to create 1 rle with old method : 0.08724188804626465 length of segment : 346 time for calcul the mask position with numpy : 0.001146078109741211 nb_pixel_total : 38929 time to create 1 rle with old method : 0.06250190734863281 length of segment : 181 time for calcul the mask position with numpy : 0.002305269241333008 nb_pixel_total : 79519 time to create 1 rle with old method : 0.08841061592102051 length of segment : 693 time for calcul the mask position with numpy : 0.004019737243652344 nb_pixel_total : 179990 time to create 1 rle with new method : 0.006551504135131836 length of segment : 469 time for calcul the mask position with numpy : 0.0007071495056152344 nb_pixel_total : 22027 time to create 1 rle with old method : 0.02556014060974121 length of segment : 179 time for calcul the mask position with numpy : 0.000997781753540039 nb_pixel_total : 35907 time to create 1 rle with old method : 0.04182124137878418 length of segment : 200 time for calcul the mask position with numpy : 0.0006189346313476562 nb_pixel_total : 27078 time to create 1 rle with old method : 0.03300762176513672 length of segment : 123 time for calcul the mask position with numpy : 0.00054168701171875 nb_pixel_total : 15774 time to create 1 rle with old method : 0.019493579864501953 length of segment : 132 time for calcul the mask position with numpy : 0.0021317005157470703 nb_pixel_total : 88144 time to create 1 rle with old method : 0.09697604179382324 length of segment : 425 time for calcul the mask position with numpy : 0.001432657241821289 nb_pixel_total : 81548 time to create 1 rle with old method : 0.09065413475036621 length of segment : 350 time for calcul the mask position with numpy : 0.001302957534790039 nb_pixel_total : 60272 time to create 1 rle with old method : 0.06818938255310059 length of segment : 225 time for calcul the mask position with numpy : 0.0011343955993652344 nb_pixel_total : 35858 time to create 1 rle with old method : 0.04176950454711914 length of segment : 315 time for calcul the mask position with numpy : 0.0002994537353515625 nb_pixel_total : 3366 time to create 1 rle with old method : 0.003785848617553711 length of segment : 117 time for calcul the mask position with numpy : 0.0006983280181884766 nb_pixel_total : 17205 time to create 1 rle with old method : 0.019983530044555664 length of segment : 204 time for calcul the mask position with numpy : 0.0024132728576660156 nb_pixel_total : 72605 time to create 1 rle with old method : 0.08045792579650879 length of segment : 391 time for calcul the mask position with numpy : 0.0007336139678955078 nb_pixel_total : 23466 time to create 1 rle with old method : 0.02669811248779297 length of segment : 147 time for calcul the mask position with numpy : 0.0005323886871337891 nb_pixel_total : 12680 time to create 1 rle with old method : 0.01495218276977539 length of segment : 187 time for calcul the mask position with numpy : 0.0012142658233642578 nb_pixel_total : 32644 time to create 1 rle with old method : 0.0372614860534668 length of segment : 387 time for calcul the mask position with numpy : 0.00017452239990234375 nb_pixel_total : 2676 time to create 1 rle with old method : 0.003361225128173828 length of segment : 69 time for calcul the mask position with numpy : 0.0002276897430419922 nb_pixel_total : 6472 time to create 1 rle with old method : 0.008175134658813477 length of segment : 96 time for calcul the mask position with numpy : 0.0001385211944580078 nb_pixel_total : 2682 time to create 1 rle with old method : 0.003206491470336914 length of segment : 78 time for calcul the mask position with numpy : 0.0004153251647949219 nb_pixel_total : 13580 time to create 1 rle with old method : 0.015614986419677734 length of segment : 179 time for calcul the mask position with numpy : 0.0036535263061523438 nb_pixel_total : 177133 time to create 1 rle with new method : 0.008188962936401367 length of segment : 720 time for calcul the mask position with numpy : 0.0023288726806640625 nb_pixel_total : 93826 time to create 1 rle with old method : 0.10844564437866211 length of segment : 514 time spent for convertir_results : 17.869747400283813 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 153 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 47066 save missing photos in datou_result : time spend for datou_step_exec : 65.79039192199707 time spend to save output : 4.935570478439331 total time spend for step 1 : 70.7259624004364 step2:crop_condition Fri Feb 21 01:47:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 153 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 13 About to insert : list_path_to_insert length 13 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 48 About to insert : list_path_to_insert length 48 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1338640259, 1338640257, 1338640256, 1338640233, 1338640231, 1338640229, 1338640187, 1338640184, 1338640182, 1338640179] Looping around the photos to save general results len do output : 84 /-3681208758Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208757Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208760Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208763Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208777Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208796Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681313464Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208693Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208705Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208716Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208738Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208753Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208742Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208743Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208752Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208774Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208791Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208806Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681313462Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208811Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208687Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208711Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208721Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208803Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208676Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208689Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208677Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208691Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208684Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208690Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208688Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681313458Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208686Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208703Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208699Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208698Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681313459Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208725Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208712Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208710Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208713Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208714Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208715Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208732Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208733Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208734Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208756Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208750Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208744Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208741Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208762Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208769Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208767Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681313460Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208766Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208764Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208771Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208776Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208781Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208782Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208780Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208801Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208798Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208800Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208797Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208799Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208802Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208805Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208819Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208820Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208813Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208818Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681313457Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208696Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208729Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208749Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208786Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208783Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208809Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208683Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208700Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208724Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208737Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681208754Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640259', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640257', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640256', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640233', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640231', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640229', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640187', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640184', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640182', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640179', None, None, None, None, None, '2601855') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 262 time used for this insertion : 0.13161706924438477 save_final save missing photos in datou_result : time spend for datou_step_exec : 39.07006311416626 time spend to save output : 0.13364195823669434 total time spend for step 2 : 39.203705072402954 step3:thcl Fri Feb 21 01:48:18 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 2 we are using the classfication for multi_thcl [2456, 2868] time to import caffe and check if the image exist : 0.00936579704284668 time to convert the images to numpy array : 0.07299590110778809 time to import caffe and check if the image exist : 0.016008853912353516 time to convert the images to numpy array : 0.07794427871704102 time to import caffe and check if the image exist : 0.012568950653076172 time to convert the images to numpy array : 0.08928465843200684 time to import caffe and check if the image exist : 0.01615309715270996 time to convert the images to numpy array : 0.10942220687866211 time to import caffe and check if the image exist : 0.015069246292114258 time to convert the images to numpy array : 0.1563422679901123 time to import caffe and check if the image exist : 0.01595163345336914 time to convert the images to numpy array : 0.15569067001342773 time to import caffe and check if the image exist : 0.012678146362304688 time to convert the images to numpy array : 0.16025018692016602 time to import caffe and check if the image exist : 0.01583552360534668 time to convert the images to numpy array : 0.16403985023498535 time to import caffe and check if the image exist : 0.014349222183227539 time to convert the images to numpy array : 0.17322993278503418 time to import caffe and check if the image exist : 0.009439945220947266 time to convert the images to numpy array : 0.36592841148376465 total time to convert the images to numpy array : 0.7922515869140625 list photo_ids error: [] list photo_ids correct : [-3681208724, -3681208737, -3681208754, -3681313457, -3681208696, -3681208729, -3681208749, -3681208786, -3681208783, -3681208809, -3681208683, -3681208700, -3681208758, -3681208757, -3681208760, -3681208763, -3681208777, -3681208796, -3681313464, -3681208693, -3681208705, -3681208800, -3681208797, -3681208799, -3681208802, -3681208805, -3681208819, -3681208820, -3681208813, -3681208818, -3681313459, -3681208725, -3681208712, -3681208710, -3681208713, -3681208714, -3681208715, -3681208732, -3681208733, -3681208766, -3681208764, -3681208771, -3681208776, -3681208781, -3681208782, -3681208780, -3681208801, -3681208798, -3681208734, -3681208756, -3681208750, -3681208744, -3681208741, -3681208762, -3681208769, -3681208767, -3681313460, -3681208716, -3681208738, -3681208753, -3681208742, -3681208743, -3681208752, -3681208774, -3681208791, -3681208806, -3681313462, -3681208811, -3681208687, -3681208711, -3681208721, -3681208803, -3681208676, -3681208689, -3681208677, -3681208691, -3681208684, -3681208690, -3681208688, -3681313458, -3681208686, -3681208703, -3681208699, -3681208698] number of photos to traite : 84 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10589 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10370 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.0173308849334717 time used to do the prediction : 0.2553558349609375 we don't save the descriptors for this thcl 2456 tagging for thcl : 2868 To do loadFromThcl(), then load ParamDescType : thcl2868 thcls : [{'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5288 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) To loadFromThcl() : net_5288 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10368 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_papier_nantes_300421 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_papier_nantes_300421 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_papier_nantes_300421 /data/models_weight/learn_papier_nantes_300421/caffemodel size_local : 44791983 size in s3 : 44791983 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:13 caffemodel already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/deploy.prototxt size_local : 17255 size in s3 : 17255 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:12 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:14 mean.npy already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/synset_words.txt size_local : 331 size in s3 : 331 create time local : 2021-08-09 05:56:00 create time in s3 : 2021-08-06 19:22:12 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_papier_nantes_300421/deploy.prototxt caffemodel_filename : /data/models_weight/learn_papier_nantes_300421/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10368 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.0932230949401855 time used to do the prediction : 0.16115999221801758 we don't save the descriptors for this thcl 2868 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 1.811981201171875e-05 save missing photos in datou_result : time spend for datou_step_exec : 10.942620992660522 time spend to save output : 0.0006015300750732422 total time spend for step 3 : 10.943222522735596 step4:merge_mask_thcl_custom Fri Feb 21 01:48:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 153 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1338640259, 1338640257, 1338640256, 1338640233, 1338640231, 1338640229, 1338640187, 1338640184, 1338640182, 1338640179] Looping around the photos to save general results len do output : 10 /1338640259Didn't retrieve data .Didn't retrieve data . /1338640257Didn't retrieve data .Didn't retrieve data . /1338640256Didn't retrieve data .Didn't retrieve data . /1338640233Didn't retrieve data .Didn't retrieve data . /1338640231Didn't retrieve data .Didn't retrieve data . /1338640229Didn't retrieve data .Didn't retrieve data . /1338640187Didn't retrieve data .Didn't retrieve data . /1338640184Didn't retrieve data .Didn't retrieve data . /1338640182Didn't retrieve data .Didn't retrieve data . /1338640179Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640259', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640257', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640256', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640233', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640231', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640229', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640187', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640184', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640182', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640179', None, None, None, None, None, '2601855') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.016845226287841797 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.22856807708740234 time spend to save output : 0.017578601837158203 total time spend for step 4 : 0.24614667892456055 step5:rle_unique_nms_with_priority Fri Feb 21 01:48:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 18 nb_hashtags : 5 time to prepare the origin masks : 10.29323434829712 time for calcul the mask position with numpy : 0.5441970825195312 nb_pixel_total : 4404222 time to create 1 rle with new method : 0.5466592311859131 time for calcul the mask position with numpy : 0.022089004516601562 nb_pixel_total : 19044 time to create 1 rle with old method : 0.021130800247192383 time for calcul the mask position with numpy : 0.023499250411987305 nb_pixel_total : 153065 time to create 1 rle with new method : 0.5321676731109619 time for calcul the mask position with numpy : 0.023403167724609375 nb_pixel_total : 19528 time to create 1 rle with old method : 0.0215606689453125 time for calcul the mask position with numpy : 0.021224260330200195 nb_pixel_total : 34921 time to create 1 rle with old method : 0.03862476348876953 time for calcul the mask position with numpy : 0.02184319496154785 nb_pixel_total : 73292 time to create 1 rle with old method : 0.0791172981262207 time for calcul the mask position with numpy : 0.022951602935791016 nb_pixel_total : 145748 time to create 1 rle with old method : 0.15704774856567383 time for calcul the mask position with numpy : 0.020498275756835938 nb_pixel_total : 6628 time to create 1 rle with old method : 0.007338285446166992 time for calcul the mask position with numpy : 0.022295475006103516 nb_pixel_total : 108229 time to create 1 rle with old method : 0.12171506881713867 time for calcul the mask position with numpy : 0.03125286102294922 nb_pixel_total : 1568229 time to create 1 rle with new method : 0.560905933380127 time for calcul the mask position with numpy : 0.021602392196655273 nb_pixel_total : 1670 time to create 1 rle with old method : 0.0022585391998291016 time for calcul the mask position with numpy : 0.021772384643554688 nb_pixel_total : 24986 time to create 1 rle with old method : 0.027403593063354492 time for calcul the mask position with numpy : 0.02570509910583496 nb_pixel_total : 73202 time to create 1 rle with old method : 0.07945442199707031 time for calcul the mask position with numpy : 0.021930217742919922 nb_pixel_total : 14052 time to create 1 rle with old method : 0.015342473983764648 time for calcul the mask position with numpy : 0.021451950073242188 nb_pixel_total : 80488 time to create 1 rle with old method : 0.08662557601928711 time for calcul the mask position with numpy : 0.022112607955932617 nb_pixel_total : 39809 time to create 1 rle with old method : 0.04334616661071777 time for calcul the mask position with numpy : 0.02218341827392578 nb_pixel_total : 125119 time to create 1 rle with old method : 0.13899922370910645 time for calcul the mask position with numpy : 0.035666704177856445 nb_pixel_total : 71204 time to create 1 rle with old method : 0.07824587821960449 time for calcul the mask position with numpy : 0.03576302528381348 nb_pixel_total : 86804 time to create 1 rle with old method : 0.09659051895141602 create new chi : 3.7203292846679688 time to delete rle : 0.01341867446899414 batch 1 Loaded 19 chid ids of type : 3726 Number RLEs to save : 16032 TO DO : save crop sub photo not yet done ! save time : 2.448094606399536 nb_obj : 14 nb_hashtags : 5 time to prepare the origin masks : 6.71773099899292 time for calcul the mask position with numpy : 0.6162436008453369 nb_pixel_total : 6159730 time to create 1 rle with new method : 0.5441246032714844 time for calcul the mask position with numpy : 0.03496956825256348 nb_pixel_total : 129438 time to create 1 rle with old method : 0.15021443367004395 time for calcul the mask position with numpy : 0.035181522369384766 nb_pixel_total : 1539 time to create 1 rle with old method : 0.0018355846405029297 time for calcul the mask position with numpy : 0.03516077995300293 nb_pixel_total : 1402 time to create 1 rle with old method : 0.0016660690307617188 time for calcul the mask position with numpy : 0.03821873664855957 nb_pixel_total : 22965 time to create 1 rle with old method : 0.024944782257080078 time for calcul the mask position with numpy : 0.03390622138977051 nb_pixel_total : 2366 time to create 1 rle with old method : 0.002615690231323242 time for calcul the mask position with numpy : 0.04073143005371094 nb_pixel_total : 379765 time to create 1 rle with new method : 0.781301736831665 time for calcul the mask position with numpy : 0.03365063667297363 nb_pixel_total : 3682 time to create 1 rle with old method : 0.004944324493408203 time for calcul the mask position with numpy : 0.03501272201538086 nb_pixel_total : 97661 time to create 1 rle with old method : 0.10852599143981934 time for calcul the mask position with numpy : 0.023183822631835938 nb_pixel_total : 2186 time to create 1 rle with old method : 0.002630949020385742 time for calcul the mask position with numpy : 0.0228731632232666 nb_pixel_total : 26461 time to create 1 rle with old method : 0.029488086700439453 time for calcul the mask position with numpy : 0.023029088973999023 nb_pixel_total : 2919 time to create 1 rle with old method : 0.0033445358276367188 time for calcul the mask position with numpy : 0.02208113670349121 nb_pixel_total : 80552 time to create 1 rle with old method : 0.09277105331420898 time for calcul the mask position with numpy : 0.02250218391418457 nb_pixel_total : 98226 time to create 1 rle with old method : 0.11079668998718262 time for calcul the mask position with numpy : 0.021998167037963867 nb_pixel_total : 41348 time to create 1 rle with old method : 0.04722476005554199 create new chi : 3.004483938217163 time to delete rle : 0.0005924701690673828 batch 1 Loaded 15 chid ids of type : 3726 Number RLEs to save : 9593 TO DO : save crop sub photo not yet done ! save time : 0.7426249980926514 nb_obj : 16 nb_hashtags : 5 time to prepare the origin masks : 9.302109718322754 time for calcul the mask position with numpy : 0.48117780685424805 nb_pixel_total : 5843364 time to create 1 rle with new method : 0.454758882522583 time for calcul the mask position with numpy : 0.03255724906921387 nb_pixel_total : 5682 time to create 1 rle with old method : 0.006321907043457031 time for calcul the mask position with numpy : 0.03425741195678711 nb_pixel_total : 41985 time to create 1 rle with old method : 0.04513120651245117 time for calcul the mask position with numpy : 0.03374958038330078 nb_pixel_total : 1573 time to create 1 rle with old method : 0.0018203258514404297 time for calcul the mask position with numpy : 0.03352975845336914 nb_pixel_total : 1113 time to create 1 rle with old method : 0.0012733936309814453 time for calcul the mask position with numpy : 0.03357362747192383 nb_pixel_total : 2888 time to create 1 rle with old method : 0.0034253597259521484 time for calcul the mask position with numpy : 0.0343470573425293 nb_pixel_total : 103397 time to create 1 rle with old method : 0.11090326309204102 time for calcul the mask position with numpy : 0.029308080673217773 nb_pixel_total : 424540 time to create 1 rle with new method : 0.6032538414001465 time for calcul the mask position with numpy : 0.021582841873168945 nb_pixel_total : 4717 time to create 1 rle with old method : 0.00592803955078125 time for calcul the mask position with numpy : 0.021605968475341797 nb_pixel_total : 42409 time to create 1 rle with old method : 0.047942399978637695 time for calcul the mask position with numpy : 0.025972604751586914 nb_pixel_total : 278338 time to create 1 rle with new method : 0.4629251956939697 time for calcul the mask position with numpy : 0.024487972259521484 nb_pixel_total : 1995 time to create 1 rle with old method : 0.002645254135131836 time for calcul the mask position with numpy : 0.022255897521972656 nb_pixel_total : 71908 time to create 1 rle with old method : 0.07703804969787598 time for calcul the mask position with numpy : 0.0211334228515625 nb_pixel_total : 5170 time to create 1 rle with old method : 0.005857229232788086 time for calcul the mask position with numpy : 0.02181410789489746 nb_pixel_total : 96159 time to create 1 rle with old method : 0.10769939422607422 time for calcul the mask position with numpy : 0.023013830184936523 nb_pixel_total : 97319 time to create 1 rle with old method : 0.1283118724822998 time for calcul the mask position with numpy : 0.022673368453979492 nb_pixel_total : 27683 time to create 1 rle with old method : 0.030213356018066406 create new chi : 3.095916271209717 time to delete rle : 0.0005793571472167969 batch 1 Loaded 17 chid ids of type : 3726 Number RLEs to save : 11874 TO DO : save crop sub photo not yet done ! save time : 0.9670815467834473 nb_obj : 14 nb_hashtags : 4 time to prepare the origin masks : 7.106290340423584 time for calcul the mask position with numpy : 0.3482801914215088 nb_pixel_total : 6599132 time to create 1 rle with new method : 0.43505191802978516 time for calcul the mask position with numpy : 0.03675508499145508 nb_pixel_total : 139720 time to create 1 rle with old method : 0.16487956047058105 time for calcul the mask position with numpy : 0.03869056701660156 nb_pixel_total : 2110 time to create 1 rle with old method : 0.0025107860565185547 time for calcul the mask position with numpy : 0.0335078239440918 nb_pixel_total : 17273 time to create 1 rle with old method : 0.018448352813720703 time for calcul the mask position with numpy : 0.03293204307556152 nb_pixel_total : 3236 time to create 1 rle with old method : 0.0034389495849609375 time for calcul the mask position with numpy : 0.03338193893432617 nb_pixel_total : 1491 time to create 1 rle with old method : 0.0017616748809814453 time for calcul the mask position with numpy : 0.0331270694732666 nb_pixel_total : 1939 time to create 1 rle with old method : 0.0022945404052734375 time for calcul the mask position with numpy : 0.03062748908996582 nb_pixel_total : 96367 time to create 1 rle with old method : 0.1068415641784668 time for calcul the mask position with numpy : 0.034635305404663086 nb_pixel_total : 6604 time to create 1 rle with old method : 0.007686614990234375 time for calcul the mask position with numpy : 0.032800912857055664 nb_pixel_total : 75637 time to create 1 rle with old method : 0.08898806571960449 time for calcul the mask position with numpy : 0.02509927749633789 nb_pixel_total : 1447 time to create 1 rle with old method : 0.0017631053924560547 time for calcul the mask position with numpy : 0.021349191665649414 nb_pixel_total : 880 time to create 1 rle with old method : 0.0013604164123535156 time for calcul the mask position with numpy : 0.0218505859375 nb_pixel_total : 75656 time to create 1 rle with old method : 0.08459854125976562 time for calcul the mask position with numpy : 0.0214998722076416 nb_pixel_total : 25914 time to create 1 rle with old method : 0.028383731842041016 time for calcul the mask position with numpy : 0.02139449119567871 nb_pixel_total : 2834 time to create 1 rle with old method : 0.0033156871795654297 create new chi : 1.7562885284423828 time to delete rle : 0.00043964385986328125 batch 1 Loaded 15 chid ids of type : 3726 Number RLEs to save : 7777 TO DO : save crop sub photo not yet done ! save time : 1.2011499404907227 nb_obj : 17 nb_hashtags : 5 time to prepare the origin masks : 8.674409627914429 time for calcul the mask position with numpy : 0.36930298805236816 nb_pixel_total : 5907533 time to create 1 rle with new method : 0.4988529682159424 time for calcul the mask position with numpy : 0.02571249008178711 nb_pixel_total : 121994 time to create 1 rle with old method : 0.13151788711547852 time for calcul the mask position with numpy : 0.021680831909179688 nb_pixel_total : 3215 time to create 1 rle with old method : 0.0037560462951660156 time for calcul the mask position with numpy : 0.022133827209472656 nb_pixel_total : 14944 time to create 1 rle with old method : 0.01635909080505371 time for calcul the mask position with numpy : 0.022313594818115234 nb_pixel_total : 2359 time to create 1 rle with old method : 0.0027947425842285156 time for calcul the mask position with numpy : 0.021257638931274414 nb_pixel_total : 171 time to create 1 rle with old method : 0.00022530555725097656 time for calcul the mask position with numpy : 0.02184152603149414 nb_pixel_total : 2318 time to create 1 rle with old method : 0.002629518508911133 time for calcul the mask position with numpy : 0.02144312858581543 nb_pixel_total : 7461 time to create 1 rle with old method : 0.008279800415039062 time for calcul the mask position with numpy : 0.023990869522094727 nb_pixel_total : 125842 time to create 1 rle with old method : 0.14180421829223633 time for calcul the mask position with numpy : 0.03389453887939453 nb_pixel_total : 1850 time to create 1 rle with old method : 0.002153158187866211 time for calcul the mask position with numpy : 0.03311753273010254 nb_pixel_total : 4642 time to create 1 rle with old method : 0.0052089691162109375 time for calcul the mask position with numpy : 0.03325390815734863 nb_pixel_total : 63670 time to create 1 rle with old method : 0.06891417503356934 time for calcul the mask position with numpy : 0.03804898262023926 nb_pixel_total : 376704 time to create 1 rle with new method : 0.5174038410186768 time for calcul the mask position with numpy : 0.03138446807861328 nb_pixel_total : 91221 time to create 1 rle with old method : 0.09835338592529297 time for calcul the mask position with numpy : 0.028417110443115234 nb_pixel_total : 129713 time to create 1 rle with old method : 0.14214110374450684 time for calcul the mask position with numpy : 0.020204544067382812 nb_pixel_total : 2267 time to create 1 rle with old method : 0.0026102066040039062 time for calcul the mask position with numpy : 0.02121734619140625 nb_pixel_total : 131420 time to create 1 rle with old method : 0.1499805450439453 time for calcul the mask position with numpy : 0.023573875427246094 nb_pixel_total : 62916 time to create 1 rle with old method : 0.06636190414428711 create new chi : 2.738163948059082 time to delete rle : 0.0004830360412597656 batch 1 Loaded 18 chid ids of type : 3726 Number RLEs to save : 10862 TO DO : save crop sub photo not yet done ! save time : 2.7620632648468018 nb_obj : 16 nb_hashtags : 6 time to prepare the origin masks : 9.866492986679077 time for calcul the mask position with numpy : 0.43073582649230957 nb_pixel_total : 6351245 time to create 1 rle with new method : 0.7920901775360107 time for calcul the mask position with numpy : 0.024823665618896484 nb_pixel_total : 121197 time to create 1 rle with old method : 0.139695405960083 time for calcul the mask position with numpy : 0.02308201789855957 nb_pixel_total : 13500 time to create 1 rle with old method : 0.015454769134521484 time for calcul the mask position with numpy : 0.023249387741088867 nb_pixel_total : 19527 time to create 1 rle with old method : 0.022654294967651367 time for calcul the mask position with numpy : 0.0232086181640625 nb_pixel_total : 58712 time to create 1 rle with old method : 0.06543326377868652 time for calcul the mask position with numpy : 0.022614717483520508 nb_pixel_total : 110750 time to create 1 rle with old method : 0.1226649284362793 time for calcul the mask position with numpy : 0.021418094635009766 nb_pixel_total : 19166 time to create 1 rle with old method : 0.021069049835205078 time for calcul the mask position with numpy : 0.021561861038208008 nb_pixel_total : 7992 time to create 1 rle with old method : 0.008938789367675781 time for calcul the mask position with numpy : 0.021244287490844727 nb_pixel_total : 8752 time to create 1 rle with old method : 0.009738922119140625 time for calcul the mask position with numpy : 0.021121978759765625 nb_pixel_total : 45062 time to create 1 rle with old method : 0.048933982849121094 time for calcul the mask position with numpy : 0.020486831665039062 nb_pixel_total : 1476 time to create 1 rle with old method : 0.0016674995422363281 time for calcul the mask position with numpy : 0.021819591522216797 nb_pixel_total : 16213 time to create 1 rle with old method : 0.018065452575683594 time for calcul the mask position with numpy : 0.02187061309814453 nb_pixel_total : 62020 time to create 1 rle with old method : 0.06693744659423828 time for calcul the mask position with numpy : 0.02128744125366211 nb_pixel_total : 33143 time to create 1 rle with old method : 0.03641557693481445 time for calcul the mask position with numpy : 0.021925687789916992 nb_pixel_total : 104588 time to create 1 rle with old method : 0.11232614517211914 time for calcul the mask position with numpy : 0.02683711051940918 nb_pixel_total : 34202 time to create 1 rle with old method : 0.0374293327331543 time for calcul the mask position with numpy : 0.021112918853759766 nb_pixel_total : 42695 time to create 1 rle with old method : 0.046671152114868164 create new chi : 2.392789840698242 time to delete rle : 0.0006062984466552734 batch 1 Loaded 17 chid ids of type : 3726 Number RLEs to save : 12573 TO DO : save crop sub photo not yet done ! save time : 1.0589327812194824 nb_obj : 5 nb_hashtags : 3 time to prepare the origin masks : 4.175199270248413 time for calcul the mask position with numpy : 0.03931856155395508 nb_pixel_total : 2255366 time to create 1 rle with new method : 0.8021969795227051 time for calcul the mask position with numpy : 0.02129673957824707 nb_pixel_total : 151046 time to create 1 rle with new method : 0.499713659286499 time for calcul the mask position with numpy : 0.021959304809570312 nb_pixel_total : 43515 time to create 1 rle with old method : 0.04792618751525879 time for calcul the mask position with numpy : 0.021892070770263672 nb_pixel_total : 2310 time to create 1 rle with old method : 0.002643108367919922 time for calcul the mask position with numpy : 0.021518707275390625 nb_pixel_total : 9610 time to create 1 rle with old method : 0.013343334197998047 time for calcul the mask position with numpy : 0.3724782466888428 nb_pixel_total : 4588393 time to create 1 rle with new method : 0.840803861618042 create new chi : 2.7858262062072754 time to delete rle : 0.0005066394805908203 batch 1 Loaded 6 chid ids of type : 3726 Number RLEs to save : 9614 TO DO : save crop sub photo not yet done ! save time : 1.4468719959259033 nb_obj : 16 nb_hashtags : 5 time to prepare the origin masks : 9.256000280380249 time for calcul the mask position with numpy : 0.38259410858154297 nb_pixel_total : 5538664 time to create 1 rle with new method : 0.8388521671295166 time for calcul the mask position with numpy : 0.02081751823425293 nb_pixel_total : 875 time to create 1 rle with old method : 0.0010707378387451172 time for calcul the mask position with numpy : 0.02114081382751465 nb_pixel_total : 14839 time to create 1 rle with old method : 0.016010284423828125 time for calcul the mask position with numpy : 0.022109031677246094 nb_pixel_total : 147262 time to create 1 rle with old method : 0.1586465835571289 time for calcul the mask position with numpy : 0.02783989906311035 nb_pixel_total : 593206 time to create 1 rle with new method : 0.5591907501220703 time for calcul the mask position with numpy : 0.03658795356750488 nb_pixel_total : 49361 time to create 1 rle with old method : 0.05986809730529785 time for calcul the mask position with numpy : 0.03406953811645508 nb_pixel_total : 16088 time to create 1 rle with old method : 0.017773151397705078 time for calcul the mask position with numpy : 0.03391861915588379 nb_pixel_total : 35235 time to create 1 rle with old method : 0.038733482360839844 time for calcul the mask position with numpy : 0.032921552658081055 nb_pixel_total : 6804 time to create 1 rle with old method : 0.00804448127746582 time for calcul the mask position with numpy : 0.03505563735961914 nb_pixel_total : 270981 time to create 1 rle with new method : 0.6747379302978516 time for calcul the mask position with numpy : 0.02828049659729004 nb_pixel_total : 9658 time to create 1 rle with old method : 0.010618448257446289 time for calcul the mask position with numpy : 0.023354768753051758 nb_pixel_total : 113277 time to create 1 rle with old method : 0.12044668197631836 time for calcul the mask position with numpy : 0.022215604782104492 nb_pixel_total : 51885 time to create 1 rle with old method : 0.05896472930908203 time for calcul the mask position with numpy : 0.021594524383544922 nb_pixel_total : 5073 time to create 1 rle with old method : 0.006144046783447266 time for calcul the mask position with numpy : 0.021207571029663086 nb_pixel_total : 139541 time to create 1 rle with old method : 0.15093684196472168 time for calcul the mask position with numpy : 0.022633075714111328 nb_pixel_total : 48656 time to create 1 rle with old method : 0.053334712982177734 time for calcul the mask position with numpy : 0.022693157196044922 nb_pixel_total : 8835 time to create 1 rle with old method : 0.009805440902709961 create new chi : 3.6788973808288574 time to delete rle : 0.0009915828704833984 batch 1 Loaded 17 chid ids of type : 3726 Number RLEs to save : 13453 TO DO : save crop sub photo not yet done ! save time : 0.9774127006530762 nb_obj : 11 nb_hashtags : 4 time to prepare the origin masks : 6.023696660995483 time for calcul the mask position with numpy : 0.3587799072265625 nb_pixel_total : 6420860 time to create 1 rle with new method : 0.45484423637390137 time for calcul the mask position with numpy : 0.031827688217163086 nb_pixel_total : 60130 time to create 1 rle with old method : 0.06506609916687012 time for calcul the mask position with numpy : 0.0340118408203125 nb_pixel_total : 1225 time to create 1 rle with old method : 0.001753091812133789 time for calcul the mask position with numpy : 0.03468728065490723 nb_pixel_total : 21811 time to create 1 rle with old method : 0.027217388153076172 time for calcul the mask position with numpy : 0.03475594520568848 nb_pixel_total : 15675 time to create 1 rle with old method : 0.017707347869873047 time for calcul the mask position with numpy : 0.035186767578125 nb_pixel_total : 80967 time to create 1 rle with old method : 0.08728265762329102 time for calcul the mask position with numpy : 0.033994436264038086 nb_pixel_total : 80145 time to create 1 rle with old method : 0.08995509147644043 time for calcul the mask position with numpy : 0.03423261642456055 nb_pixel_total : 26913 time to create 1 rle with old method : 0.02958083152770996 time for calcul the mask position with numpy : 0.03214526176452637 nb_pixel_total : 38810 time to create 1 rle with old method : 0.042452335357666016 time for calcul the mask position with numpy : 0.03335905075073242 nb_pixel_total : 179939 time to create 1 rle with new method : 0.4954531192779541 time for calcul the mask position with numpy : 0.02100658416748047 nb_pixel_total : 35849 time to create 1 rle with old method : 0.03912854194641113 time for calcul the mask position with numpy : 0.02105545997619629 nb_pixel_total : 87916 time to create 1 rle with old method : 0.0954580307006836 create new chi : 2.2141411304473877 time to delete rle : 0.00047707557678222656 batch 1 Loaded 12 chid ids of type : 3726 Number RLEs to save : 8394 TO DO : save crop sub photo not yet done ! save time : 0.7810359001159668 nb_obj : 14 nb_hashtags : 4 time to prepare the origin masks : 8.46021056175232 time for calcul the mask position with numpy : 0.6656303405761719 nb_pixel_total : 6556633 time to create 1 rle with new method : 0.4981350898742676 time for calcul the mask position with numpy : 0.021212100982666016 nb_pixel_total : 72418 time to create 1 rle with old method : 0.07895922660827637 time for calcul the mask position with numpy : 0.020400285720825195 nb_pixel_total : 17152 time to create 1 rle with old method : 0.018694162368774414 time for calcul the mask position with numpy : 0.034477949142456055 nb_pixel_total : 12596 time to create 1 rle with old method : 0.013395309448242188 time for calcul the mask position with numpy : 0.032715797424316406 nb_pixel_total : 13549 time to create 1 rle with old method : 0.014461040496826172 time for calcul the mask position with numpy : 0.033331871032714844 nb_pixel_total : 93751 time to create 1 rle with old method : 0.09681129455566406 time for calcul the mask position with numpy : 0.03299236297607422 nb_pixel_total : 2667 time to create 1 rle with old method : 0.0032286643981933594 time for calcul the mask position with numpy : 0.030187129974365234 nb_pixel_total : 6454 time to create 1 rle with old method : 0.006970643997192383 time for calcul the mask position with numpy : 0.032629966735839844 nb_pixel_total : 2674 time to create 1 rle with old method : 0.0031545162200927734 time for calcul the mask position with numpy : 0.031296730041503906 nb_pixel_total : 31872 time to create 1 rle with old method : 0.035146474838256836 time for calcul the mask position with numpy : 0.026200532913208008 nb_pixel_total : 22408 time to create 1 rle with old method : 0.02480340003967285 time for calcul the mask position with numpy : 0.02084803581237793 nb_pixel_total : 3370 time to create 1 rle with old method : 0.00393366813659668 time for calcul the mask position with numpy : 0.02044081687927246 nb_pixel_total : 1692 time to create 1 rle with old method : 0.0020453929901123047 time for calcul the mask position with numpy : 0.020703792572021484 nb_pixel_total : 35801 time to create 1 rle with old method : 0.03852653503417969 time for calcul the mask position with numpy : 0.022734642028808594 nb_pixel_total : 177203 time to create 1 rle with new method : 0.4103395938873291 create new chi : 2.358577251434326 time to delete rle : 0.0004534721374511719 batch 1 Loaded 15 chid ids of type : 3726 Number RLEs to save : 9244 TO DO : save crop sub photo not yet done ! save time : 0.7779557704925537 map_output_result : {1338640259: (0.0, 'Should be the crop_list due to order', 0), 1338640257: (0.0, 'Should be the crop_list due to order', 0), 1338640256: (0.0, 'Should be the crop_list due to order', 0), 1338640233: (0.0, 'Should be the crop_list due to order', 0), 1338640231: (0.0, 'Should be the crop_list due to order', 0), 1338640229: (0.0, 'Should be the crop_list due to order', 0), 1338640187: (0.0, 'Should be the crop_list due to order', 0), 1338640184: (0.0, 'Should be the crop_list due to order', 0), 1338640182: (0.0, 'Should be the crop_list due to order', 0), 1338640179: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1338640259, 1338640257, 1338640256, 1338640233, 1338640231, 1338640229, 1338640187, 1338640184, 1338640182, 1338640179] Looping around the photos to save general results len do output : 10 /1338640259.Didn't retrieve data . /1338640257.Didn't retrieve data . /1338640256.Didn't retrieve data . /1338640233.Didn't retrieve data . /1338640231.Didn't retrieve data . /1338640229.Didn't retrieve data . /1338640187.Didn't retrieve data . /1338640184.Didn't retrieve data . /1338640182.Didn't retrieve data . /1338640179.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640259', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640257', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640256', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640233', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640231', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640229', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640187', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640184', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640182', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640179', None, None, None, None, None, '2601855') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.013392448425292969 save_final save missing photos in datou_result : time spend for datou_step_exec : 121.8173291683197 time spend to save output : 0.013817548751831055 total time spend for step 5 : 121.83114671707153 step6:crop_condition Fri Feb 21 01:50:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 151 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 4869462 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740099034_336713 we have uploaded 2 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.9922189712524414 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 4869462 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740099037_336713 we have uploaded 4 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.7843096256256104 we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 4869462 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740099045_336713 we have uploaded 5 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.7728254795074463 we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 4869462 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740099050_336713 we have uploaded 5 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.7915794849395752 we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 4869462 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740099054_336713 we have uploaded 2 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.8220827579498291 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! About to upload 7 photos upload in portfolio : 4869462 init cache_photo without model_param we have 7 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740099059_336713 we have uploaded 7 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.1524507999420166 we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1338640259, 1338640257, 1338640256, 1338640233, 1338640231, 1338640229, 1338640187, 1338640184, 1338640182, 1338640179] Looping around the photos to save general results len do output : 25 /1338711994Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338711995Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338711998Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338711999Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338712000Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338712001Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338712004Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338712005Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338712006Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338712007Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338712008Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338712011Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338712012Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338712013Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338712014Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338712015Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338712019Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338712020Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338712023Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338712024Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338712025Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338712026Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338712027Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338712028Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338712029Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640259', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640257', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640256', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640233', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640231', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640229', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640187', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640184', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640182', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640179', None, None, None, None, None, '2601855') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 85 time used for this insertion : 0.035086631774902344 save_final save missing photos in datou_result : time spend for datou_step_exec : 31.220383405685425 time spend to save output : 0.035909414291381836 total time spend for step 6 : 31.256292819976807 step7:ventilate_hashtags_in_portfolio Fri Feb 21 01:51:03 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 20699439 get user id for portfolio 20699439 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20699439 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('autre_refus','Carton_gris','environnement','Teint_Dans_La_Masse','plastique','flou','cartonnette','papier','kraft','metal','Carton_brun','mal_croppe')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20699439 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('autre_refus','Carton_gris','environnement','Teint_Dans_La_Masse','plastique','flou','cartonnette','papier','kraft','metal','Carton_brun','mal_croppe')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/20699694,20699695,20699696,20699697,20699698,20699699,20699700,20699701,20699702,20699703,20699704,20699705?tags=autre_refus,plastique,Carton_gris,mal_croppe,Teint_Dans_La_Masse,environnement,flou,Carton_brun,metal,papier,kraft,cartonnette Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1338640259, 1338640257, 1338640256, 1338640233, 1338640231, 1338640229, 1338640187, 1338640184, 1338640182, 1338640179] Looping around the photos to save general results len do output : 1 /20699439. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640259', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640257', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640256', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640233', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640231', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640229', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640187', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640184', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640182', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640179', None, None, None, None, None, '2601855') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.01434636116027832 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.6260263919830322 time spend to save output : 0.014721870422363281 total time spend for step 7 : 1.6407482624053955 step8:final Fri Feb 21 01:51:04 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1338640259: ('0.672679637942406',), 1338640257: ('0.672679637942406',), 1338640256: ('0.672679637942406',), 1338640233: ('0.672679637942406',), 1338640231: ('0.672679637942406',), 1338640229: ('0.672679637942406',), 1338640187: ('0.672679637942406',), 1338640184: ('0.672679637942406',), 1338640182: ('0.672679637942406',), 1338640179: ('0.672679637942406',)} new output for save of step final : {1338640259: ('0.672679637942406',), 1338640257: ('0.672679637942406',), 1338640256: ('0.672679637942406',), 1338640233: ('0.672679637942406',), 1338640231: ('0.672679637942406',), 1338640229: ('0.672679637942406',), 1338640187: ('0.672679637942406',), 1338640184: ('0.672679637942406',), 1338640182: ('0.672679637942406',), 1338640179: ('0.672679637942406',)} [1338640259, 1338640257, 1338640256, 1338640233, 1338640231, 1338640229, 1338640187, 1338640184, 1338640182, 1338640179] Looping around the photos to save general results len do output : 10 /1338640259.Didn't retrieve data . /1338640257.Didn't retrieve data . /1338640256.Didn't retrieve data . /1338640233.Didn't retrieve data . /1338640231.Didn't retrieve data . /1338640229.Didn't retrieve data . /1338640187.Didn't retrieve data . /1338640184.Didn't retrieve data . /1338640182.Didn't retrieve data . /1338640179.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640259', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640257', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640256', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640233', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640231', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640229', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640187', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640184', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640182', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640179', None, None, None, None, None, '2601855') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.013944149017333984 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.10763192176818848 time spend to save output : 0.014427661895751953 total time spend for step 8 : 0.12205958366394043 step9:velours_tree Fri Feb 21 01:51:04 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.03721427917480469 time spend to save output : 4.220008850097656e-05 total time spend for step 9 : 0.037256479263305664 step10:send_mail_cod Fri Feb 21 01:51:05 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P20699439_21-02-2025_01_51_05.pdf 20699694 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette206996941740099065 20699695 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette206996951740099065 20699697 imagette206996971740099066 20699700 imagette206997001740099066 20699702 imagette206997021740099066 20699704 imagette206997041740099066 20699705 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette206997051740099066 20699696 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette206996961740099067 20699698 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette206996981740099068 20699701 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette206997011740099069 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20699439 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20699694,20699695,20699696,20699697,20699698,20699699,20699700,20699701,20699702,20699703,20699704,20699705?tags=autre_refus,plastique,Carton_gris,mal_croppe,Teint_Dans_La_Masse,environnement,flou,Carton_brun,metal,papier,kraft,cartonnette your option no_mail is active, we will not send the real mail to your client args[1338640259] : ('0.672679637942406',) no score found for photo 1338640259 We are sending mail with results at report@fotonower.com args[1338640257] : ('0.672679637942406',) no score found for photo 1338640257 We are sending mail with results at report@fotonower.com args[1338640256] : ('0.672679637942406',) no score found for photo 1338640256 We are sending mail with results at report@fotonower.com args[1338640233] : ('0.672679637942406',) no score found for photo 1338640233 We are sending mail with results at report@fotonower.com args[1338640231] : ('0.672679637942406',) no score found for photo 1338640231 We are sending mail with results at report@fotonower.com args[1338640229] : ('0.672679637942406',) no score found for photo 1338640229 We are sending mail with results at report@fotonower.com args[1338640187] : ('0.672679637942406',) no score found for photo 1338640187 We are sending mail with results at report@fotonower.com args[1338640184] : ('0.672679637942406',) no score found for photo 1338640184 We are sending mail with results at report@fotonower.com args[1338640182] : ('0.672679637942406',) no score found for photo 1338640182 We are sending mail with results at report@fotonower.com args[1338640179] : ('0.672679637942406',) no score found for photo 1338640179 We are sending mail with results at report@fotonower.com refus_total : 0.672679637942406 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20699439 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1338640177,1338690108,1338690104,1338690094,1338690091,1338640318,1338640277,1338640257,1338640256,1338640233,1338640231,1338640229,1338640184,1338640182,1338640179,1338640175,1338640275,1338640259,1338640187,1338690078) Found this number of photos: 20 begin to download photo : 1338640177 begin to download photo : 1338640318 begin to download photo : 1338640231 begin to download photo : 1338640175 download finish for photo 1338640177 begin to download photo : 1338690108 download finish for photo 1338640231 begin to download photo : 1338640229 download finish for photo 1338640175 begin to download photo : 1338640275 download finish for photo 1338640229 begin to download photo : 1338640184 download finish for photo 1338640275 begin to download photo : 1338640259 download finish for photo 1338690108 begin to download photo : 1338690104 download finish for photo 1338640318 begin to download photo : 1338640277 download finish for photo 1338640184 begin to download photo : 1338640182 download finish for photo 1338640259 begin to download photo : 1338640187 download finish for photo 1338690104 begin to download photo : 1338690094 download finish for photo 1338640277 begin to download photo : 1338640257 download finish for photo 1338640182 begin to download photo : 1338640179 download finish for photo 1338690094 begin to download photo : 1338690091 download finish for photo 1338640187 begin to download photo : 1338690078 download finish for photo 1338640257 begin to download photo : 1338640256 download finish for photo 1338640179 download finish for photo 1338690091 download finish for photo 1338640256 begin to download photo : 1338640233 download finish for photo 1338690078 download finish for photo 1338640233 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20699439_21-02-2025_01_51_05.pdf results_Auto_P20699439_21-02-2025_01_51_05.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20699439_21-02-2025_01_51_05.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3459','20699439','results_Auto_P20699439_21-02-2025_01_51_05.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20699439_21-02-2025_01_51_05.pdf','pdf','','1.07','0.672679637942406') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1338640259, 1338640257, 1338640256, 1338640233, 1338640231, 1338640229, 1338640187, 1338640184, 1338640182, 1338640179] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640259', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640257', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640256', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640233', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640231', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640229', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640187', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640184', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640182', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640179', None, None, None, None, None, '2601855') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.18628907203674316 save_final save missing photos in datou_result : time spend for datou_step_exec : 9.731883525848389 time spend to save output : 0.18662333488464355 total time spend for step 10 : 9.918506860733032 step11:split_time_score Fri Feb 21 01:51:14 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('17', 22),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 20022025 20699439 Nombre de photos uploadées : 22 / 23040 (0%) 20022025 20699439 Nombre de photos taguées (types de déchets): 0 / 22 (0%) 20022025 20699439 Nombre de photos taguées (volume) : 0 / 22 (0%) elapsed_time : load_data_split_time_score 1.430511474609375e-06 elapsed_time : order_list_meta_photo_and_scores 5.245208740234375e-06 ?????????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0011248588562011719 elapsed_time : insert_dashboard_record_day_entry 0.027750492095947266 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.9608671591978778 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20699402_21-02-2025_00_49_49.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699402 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20699402 AND mptpi.`type`=3726 To do Qualite : 0.6267786478778448 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20699403_21-02-2025_00_45_40.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699403 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20699403 AND mptpi.`type`=3726 To do Qualite : 0.0 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20699404_21-02-2025_01_35_14.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699404 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20699404 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699405 order by id desc limit 1 Qualite : 0.3329848280211983 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20699406_21-02-2025_00_39_22.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699406 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20699406 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699407 order by id desc limit 1 Qualite : 0.16317802656363475 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20699408_21-02-2025_01_07_21.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699408 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20699408 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699409 order by id desc limit 1 Qualite : 0.06501049850080148 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20699410_21-02-2025_01_32_35.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699410 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11524 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11533 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11532 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11528 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11528 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11531 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11531 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11525 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11525 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11578 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11578 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11527 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11526 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11526 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11535 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11527 doesn't seem to be define in the database( WARNING : type of input 3 of step 11526 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11524 doesn't seem to be define in the database( WARNING : type of input 2 of step 11528 doesn't seem to be define in the database( WARNING : output 1 of step 11524 have datatype=2 whereas input 1 of step 11531 have datatype=7 WARNING : type of output 2 of step 11531 doesn't seem to be define in the database( WARNING : type of input 1 of step 11525 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11527 have datatype=10 whereas input 3 of step 11534 have datatype=6 WARNING : type of input 2 of step 11578 doesn't seem to be define in the database( WARNING : output 1 of step 11525 have datatype=7 whereas input 2 of step 11578 have datatype=None WARNING : type of output 3 of step 11578 doesn't seem to be define in the database( WARNING : type of input 1 of step 11527 doesn't seem to be define in the database( WARNING : output 0 of step 11527 have datatype=10 whereas input 0 of step 11584 have datatype=18 WARNING : type of input 5 of step 11534 doesn't seem to be define in the database( WARNING : output 0 of step 11584 have datatype=11 whereas input 5 of step 11534 have datatype=None WARNING : type of output 1 of step 11533 doesn't seem to be define in the database( WARNING : type of input 3 of step 11528 doesn't seem to be define in the database( WARNING : type of output 1 of step 11532 doesn't seem to be define in the database( WARNING : type of input 3 of step 11528 doesn't seem to be define in the database( WARNING : output 0 of step 11531 have datatype=1 whereas input 0 of step 11525 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20699410 AND mptpi.`type`=4211 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699411 order by id desc limit 1 Qualite : 0.6814549659204545 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20699412_21-02-2025_00_35_10.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699412 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20699412 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699413 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699414 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699415 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699416 order by id desc limit 1 Qualite : 0.3767832473441416 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20699417_21-02-2025_00_19_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699417 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20699417 AND mptpi.`type`=4207 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699418 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699419 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699420 order by id desc limit 1 Qualite : 0.20185023706776162 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20699421_21-02-2025_01_06_25.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699421 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20699421 AND mptpi.`type`=3594 To do Qualite : 0.21633741933221104 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20699422_21-02-2025_00_52_59.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699422 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20699422 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699423 order by id desc limit 1 Qualite : 1.0 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20699424_21-02-2025_01_31_08.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699424 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20699424 AND mptpi.`type`=4200 To do Qualite : 0.11956076831146291 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20699425_21-02-2025_01_43_51.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699425 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20699425 AND mptpi.`type`=3327 To do Qualite : 0.021452326498044953 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20699426_21-02-2025_01_17_58.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699426 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20699426 AND mptpi.`type`=3327 To do Qualite : 0.7519938324117397 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20699439_21-02-2025_01_51_05.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699439 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20699439 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699440 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699441 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699442 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699443 order by id desc limit 1 Qualite : 0.9623625257041976 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20699444_21-02-2025_01_40_57.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699444 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20699444 AND mptpi.`type`=4207 To do Qualite : 0.2402460767293029 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20699449_21-02-2025_00_43_32.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699449 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20699449 AND mptpi.`type`=3594 To do Qualite : 0.16941512806656225 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20699470_21-02-2025_00_37_56.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699470 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20699470 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699471 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699472 order by id desc limit 1 Qualite : 0.9703251001887719 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20699473_21-02-2025_01_39_38.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699473 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20699473 AND mptpi.`type`=4207 To do Qualite : 0.8805410694113283 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20699474_21-02-2025_01_03_29.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699474 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20699474 AND mptpi.`type`=4207 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'20022025': {'nb_upload': 22, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1338640259, 1338640257, 1338640256, 1338640233, 1338640231, 1338640229, 1338640187, 1338640184, 1338640182, 1338640179] Looping around the photos to save general results len do output : 1 /20699439Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640259', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640257', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640256', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640233', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640231', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640229', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640187', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640184', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640182', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', None, '1338640179', None, None, None, None, None, '2601855') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.022242069244384766 save_final save missing photos in datou_result : time spend for datou_step_exec : 16.393097400665283 time spend to save output : 0.022487878799438477 total time spend for step 11 : 16.41558527946472 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 143.52user 118.43system 5:07.40elapsed 85%CPU (0avgtext+0avgdata 4260076maxresident)k 998528inputs+139200outputs (2778major+11928744minor)pagefaults 0swaps