python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 3459 -l 10 ' -s datou_current_3459 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 76450 load datou : 3459 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) load thcls load THCL from format json or kwargs add thcl : 2456 in CacheModelConfig load THCL from format json or kwargs add thcl : 2868 in CacheModelConfig load THCL from format json or kwargs add thcl : 2896 in CacheModelConfig load pdts add pdt : 4999 in CacheModelConfig add pdt : 5288 in CacheModelConfig add pdt : 5309 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 we have a portfolio with more photos than limit : 26>10 please execute split_portfolio.py -i 20881947 -l 10 size over we load limit photo not treated list_input_json: [] Current got : datou_id : 3459, datou_cur_ids : ['2619767'] with mtr_portfolio_ids : ['20881947'] and first list_photo_ids : [] new path : /proc/76450/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, thcl, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 10 ; length of list_pids : 10 ; length of list_args : 10 time to download the photos : 1.85953950881958 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Thu Feb 27 16:02:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10814 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-27 16:02:33.101879: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-27 16:02:33.135163: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-27 16:02:33.137560: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f516c000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-27 16:02:33.137611: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-27 16:02:33.143429: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-27 16:02:33.403166: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x13d01860 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-27 16:02:33.403220: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-27 16:02:33.404857: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-27 16:02:33.406535: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-27 16:02:33.439461: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-27 16:02:33.459949: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-27 16:02:33.464388: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-27 16:02:33.502048: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-27 16:02:33.508321: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-27 16:02:33.569387: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-27 16:02:33.571318: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-27 16:02:33.571738: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-27 16:02:33.573627: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-27 16:02:33.573654: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-27 16:02:33.573666: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-27 16:02:33.575790: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10023 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-27 16:02:33.953290: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-27 16:02:33.953431: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-27 16:02:33.953456: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-27 16:02:33.953478: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-27 16:02:33.953499: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-27 16:02:33.953520: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-27 16:02:33.953541: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-27 16:02:33.953566: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-27 16:02:33.955301: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-27 16:02:33.956704: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-27 16:02:33.956748: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-27 16:02:33.956770: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-27 16:02:33.956791: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-27 16:02:33.956811: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-27 16:02:33.956831: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-27 16:02:33.956851: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-27 16:02:33.956871: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-27 16:02:33.958593: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-27 16:02:33.958629: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-27 16:02:33.958641: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-27 16:02:33.958651: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-27 16:02:33.960400: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10023 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-27 16:02:44.593445: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-27 16:02:45.366490: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 10 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.96557 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 13 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 23 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 25 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 19 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 13 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 19 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 13 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 23 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 31 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 29 Detection mask done ! Trying to reset tf kernel 76942 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5525 tf kernel not reseted sub process len(results) : 10 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 10 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10814 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.007125377655029297 nb_pixel_total : 166316 time to create 1 rle with new method : 0.011394739151000977 length of segment : 575 time for calcul the mask position with numpy : 0.007942676544189453 nb_pixel_total : 240502 time to create 1 rle with new method : 0.01456594467163086 length of segment : 611 time for calcul the mask position with numpy : 0.45551085472106934 nb_pixel_total : 1321884 time to create 1 rle with new method : 0.1057596206665039 length of segment : 2050 time for calcul the mask position with numpy : 0.01564955711364746 nb_pixel_total : 588952 time to create 1 rle with new method : 0.6917915344238281 length of segment : 1213 time for calcul the mask position with numpy : 0.0004107952117919922 nb_pixel_total : 7217 time to create 1 rle with old method : 0.009304285049438477 length of segment : 108 time for calcul the mask position with numpy : 0.001386880874633789 nb_pixel_total : 40576 time to create 1 rle with old method : 0.06982874870300293 length of segment : 207 time for calcul the mask position with numpy : 0.0004420280456542969 nb_pixel_total : 14396 time to create 1 rle with old method : 0.02021646499633789 length of segment : 135 time for calcul the mask position with numpy : 0.0016198158264160156 nb_pixel_total : 27199 time to create 1 rle with old method : 0.03481936454772949 length of segment : 411 time for calcul the mask position with numpy : 0.0017850399017333984 nb_pixel_total : 77701 time to create 1 rle with old method : 0.08990740776062012 length of segment : 284 time for calcul the mask position with numpy : 0.007291078567504883 nb_pixel_total : 290458 time to create 1 rle with new method : 0.013591289520263672 length of segment : 901 time for calcul the mask position with numpy : 0.001177072525024414 nb_pixel_total : 28588 time to create 1 rle with old method : 0.03493785858154297 length of segment : 229 time for calcul the mask position with numpy : 0.0021173954010009766 nb_pixel_total : 43665 time to create 1 rle with old method : 0.053601741790771484 length of segment : 411 time for calcul the mask position with numpy : 0.0021817684173583984 nb_pixel_total : 60271 time to create 1 rle with old method : 0.0683903694152832 length of segment : 328 time for calcul the mask position with numpy : 0.002572298049926758 nb_pixel_total : 61055 time to create 1 rle with old method : 0.07749199867248535 length of segment : 423 time for calcul the mask position with numpy : 0.003632068634033203 nb_pixel_total : 99664 time to create 1 rle with old method : 0.1346902847290039 length of segment : 496 time for calcul the mask position with numpy : 0.0004627704620361328 nb_pixel_total : 5432 time to create 1 rle with old method : 0.007765054702758789 length of segment : 119 time for calcul the mask position with numpy : 0.0022699832916259766 nb_pixel_total : 58219 time to create 1 rle with old method : 0.0689687728881836 length of segment : 334 time for calcul the mask position with numpy : 0.0001201629638671875 nb_pixel_total : 1502 time to create 1 rle with old method : 0.001928567886352539 length of segment : 83 time for calcul the mask position with numpy : 0.00020003318786621094 nb_pixel_total : 2664 time to create 1 rle with old method : 0.003367900848388672 length of segment : 117 time for calcul the mask position with numpy : 0.0007576942443847656 nb_pixel_total : 25434 time to create 1 rle with old method : 0.031940460205078125 length of segment : 160 time for calcul the mask position with numpy : 0.0011332035064697266 nb_pixel_total : 29735 time to create 1 rle with old method : 0.03523540496826172 length of segment : 188 time for calcul the mask position with numpy : 0.0008335113525390625 nb_pixel_total : 27119 time to create 1 rle with old method : 0.033118247985839844 length of segment : 213 time for calcul the mask position with numpy : 0.0009319782257080078 nb_pixel_total : 17526 time to create 1 rle with old method : 0.02208566665649414 length of segment : 285 time for calcul the mask position with numpy : 0.0005342960357666016 nb_pixel_total : 9199 time to create 1 rle with old method : 0.011153221130371094 length of segment : 123 time for calcul the mask position with numpy : 0.001054525375366211 nb_pixel_total : 27272 time to create 1 rle with old method : 0.032514095306396484 length of segment : 228 time for calcul the mask position with numpy : 0.0008161067962646484 nb_pixel_total : 34113 time to create 1 rle with old method : 0.04164433479309082 length of segment : 221 time for calcul the mask position with numpy : 0.00043582916259765625 nb_pixel_total : 7412 time to create 1 rle with old method : 0.009546041488647461 length of segment : 270 time for calcul the mask position with numpy : 0.00023627281188964844 nb_pixel_total : 5079 time to create 1 rle with old method : 0.0062830448150634766 length of segment : 92 time for calcul the mask position with numpy : 9.965896606445312e-05 nb_pixel_total : 2648 time to create 1 rle with old method : 0.0035479068756103516 length of segment : 72 time for calcul the mask position with numpy : 0.0009932518005371094 nb_pixel_total : 20128 time to create 1 rle with old method : 0.025611400604248047 length of segment : 306 time for calcul the mask position with numpy : 0.002109050750732422 nb_pixel_total : 59240 time to create 1 rle with old method : 0.10013484954833984 length of segment : 411 time for calcul the mask position with numpy : 0.0011944770812988281 nb_pixel_total : 27427 time to create 1 rle with old method : 0.04805350303649902 length of segment : 223 time for calcul the mask position with numpy : 0.0004551410675048828 nb_pixel_total : 10682 time to create 1 rle with old method : 0.013697624206542969 length of segment : 124 time for calcul the mask position with numpy : 0.0022115707397460938 nb_pixel_total : 72382 time to create 1 rle with old method : 0.0823512077331543 length of segment : 412 time for calcul the mask position with numpy : 0.0019168853759765625 nb_pixel_total : 49585 time to create 1 rle with old method : 0.06060314178466797 length of segment : 385 time for calcul the mask position with numpy : 0.0012471675872802734 nb_pixel_total : 39220 time to create 1 rle with old method : 0.04686760902404785 length of segment : 183 time for calcul the mask position with numpy : 0.001984119415283203 nb_pixel_total : 56266 time to create 1 rle with old method : 0.0655982494354248 length of segment : 411 time for calcul the mask position with numpy : 0.0027260780334472656 nb_pixel_total : 65702 time to create 1 rle with old method : 0.07504892349243164 length of segment : 457 time for calcul the mask position with numpy : 0.0007061958312988281 nb_pixel_total : 19063 time to create 1 rle with old method : 0.022663354873657227 length of segment : 204 time for calcul the mask position with numpy : 0.002046346664428711 nb_pixel_total : 38780 time to create 1 rle with old method : 0.04505300521850586 length of segment : 373 time for calcul the mask position with numpy : 0.0013337135314941406 nb_pixel_total : 42720 time to create 1 rle with old method : 0.048360586166381836 length of segment : 249 time for calcul the mask position with numpy : 0.002255678176879883 nb_pixel_total : 62402 time to create 1 rle with old method : 0.07012438774108887 length of segment : 322 time for calcul the mask position with numpy : 0.0028944015502929688 nb_pixel_total : 83704 time to create 1 rle with old method : 0.09760594367980957 length of segment : 337 time for calcul the mask position with numpy : 0.0008797645568847656 nb_pixel_total : 18431 time to create 1 rle with old method : 0.021535396575927734 length of segment : 277 time for calcul the mask position with numpy : 0.0014700889587402344 nb_pixel_total : 21771 time to create 1 rle with old method : 0.0274658203125 length of segment : 333 time for calcul the mask position with numpy : 0.0008985996246337891 nb_pixel_total : 29748 time to create 1 rle with old method : 0.03950619697570801 length of segment : 232 time for calcul the mask position with numpy : 0.0017235279083251953 nb_pixel_total : 44988 time to create 1 rle with old method : 0.05395865440368652 length of segment : 318 time for calcul the mask position with numpy : 0.0010714530944824219 nb_pixel_total : 53273 time to create 1 rle with old method : 0.06727719306945801 length of segment : 341 time for calcul the mask position with numpy : 0.004948854446411133 nb_pixel_total : 170238 time to create 1 rle with new method : 0.011119365692138672 length of segment : 1052 time for calcul the mask position with numpy : 0.0002288818359375 nb_pixel_total : 1258 time to create 1 rle with old method : 0.0017333030700683594 length of segment : 52 time for calcul the mask position with numpy : 0.0003185272216796875 nb_pixel_total : 9767 time to create 1 rle with old method : 0.012080669403076172 length of segment : 156 time for calcul the mask position with numpy : 0.005080461502075195 nb_pixel_total : 217270 time to create 1 rle with new method : 0.00821065902709961 length of segment : 444 time for calcul the mask position with numpy : 0.0003833770751953125 nb_pixel_total : 3895 time to create 1 rle with old method : 0.005762577056884766 length of segment : 193 time for calcul the mask position with numpy : 0.0013027191162109375 nb_pixel_total : 29247 time to create 1 rle with old method : 0.03888535499572754 length of segment : 241 time for calcul the mask position with numpy : 0.0018732547760009766 nb_pixel_total : 46271 time to create 1 rle with old method : 0.053687095642089844 length of segment : 411 time for calcul the mask position with numpy : 0.0038602352142333984 nb_pixel_total : 108060 time to create 1 rle with old method : 0.11818051338195801 length of segment : 624 time for calcul the mask position with numpy : 0.0020809173583984375 nb_pixel_total : 61279 time to create 1 rle with old method : 0.07195639610290527 length of segment : 417 time for calcul the mask position with numpy : 0.00193023681640625 nb_pixel_total : 58037 time to create 1 rle with old method : 0.07256627082824707 length of segment : 339 time for calcul the mask position with numpy : 0.0011081695556640625 nb_pixel_total : 42006 time to create 1 rle with old method : 0.0498509407043457 length of segment : 193 time for calcul the mask position with numpy : 0.0002162456512451172 nb_pixel_total : 2477 time to create 1 rle with old method : 0.003150463104248047 length of segment : 147 time for calcul the mask position with numpy : 0.000812530517578125 nb_pixel_total : 29862 time to create 1 rle with old method : 0.03451943397521973 length of segment : 236 time for calcul the mask position with numpy : 0.0003464221954345703 nb_pixel_total : 4972 time to create 1 rle with old method : 0.0069065093994140625 length of segment : 113 time for calcul the mask position with numpy : 0.0004589557647705078 nb_pixel_total : 9130 time to create 1 rle with old method : 0.011931180953979492 length of segment : 121 time for calcul the mask position with numpy : 0.0019524097442626953 nb_pixel_total : 55765 time to create 1 rle with old method : 0.06242251396179199 length of segment : 371 time for calcul the mask position with numpy : 0.0006833076477050781 nb_pixel_total : 5166 time to create 1 rle with old method : 0.00674128532409668 length of segment : 165 time for calcul the mask position with numpy : 0.0009098052978515625 nb_pixel_total : 29878 time to create 1 rle with old method : 0.03614640235900879 length of segment : 195 time for calcul the mask position with numpy : 0.0001544952392578125 nb_pixel_total : 2264 time to create 1 rle with old method : 0.0029561519622802734 length of segment : 56 time for calcul the mask position with numpy : 0.0005548000335693359 nb_pixel_total : 25331 time to create 1 rle with old method : 0.03029489517211914 length of segment : 224 time for calcul the mask position with numpy : 0.0007925033569335938 nb_pixel_total : 30025 time to create 1 rle with old method : 0.03594827651977539 length of segment : 212 time for calcul the mask position with numpy : 0.00023651123046875 nb_pixel_total : 4754 time to create 1 rle with old method : 0.006132602691650391 length of segment : 69 time for calcul the mask position with numpy : 0.00017261505126953125 nb_pixel_total : 1331 time to create 1 rle with old method : 0.0016312599182128906 length of segment : 85 time for calcul the mask position with numpy : 0.0008447170257568359 nb_pixel_total : 36805 time to create 1 rle with old method : 0.04344487190246582 length of segment : 224 time for calcul the mask position with numpy : 0.0013697147369384766 nb_pixel_total : 66170 time to create 1 rle with old method : 0.08402419090270996 length of segment : 246 time for calcul the mask position with numpy : 0.0007486343383789062 nb_pixel_total : 20212 time to create 1 rle with old method : 0.02835869789123535 length of segment : 136 time for calcul the mask position with numpy : 0.0014362335205078125 nb_pixel_total : 48365 time to create 1 rle with old method : 0.05907011032104492 length of segment : 360 time for calcul the mask position with numpy : 0.0023064613342285156 nb_pixel_total : 89825 time to create 1 rle with old method : 0.10297179222106934 length of segment : 345 time for calcul the mask position with numpy : 0.00042319297790527344 nb_pixel_total : 14887 time to create 1 rle with old method : 0.01889944076538086 length of segment : 115 time for calcul the mask position with numpy : 0.0025839805603027344 nb_pixel_total : 118318 time to create 1 rle with old method : 0.13504624366760254 length of segment : 472 time for calcul the mask position with numpy : 0.00022172927856445312 nb_pixel_total : 3362 time to create 1 rle with old method : 0.004157543182373047 length of segment : 76 time for calcul the mask position with numpy : 0.0009489059448242188 nb_pixel_total : 26404 time to create 1 rle with old method : 0.039526939392089844 length of segment : 213 time for calcul the mask position with numpy : 0.0016529560089111328 nb_pixel_total : 58227 time to create 1 rle with old method : 0.07246136665344238 length of segment : 224 time for calcul the mask position with numpy : 0.00018930435180664062 nb_pixel_total : 5739 time to create 1 rle with old method : 0.007683277130126953 length of segment : 45 time for calcul the mask position with numpy : 0.0014600753784179688 nb_pixel_total : 18267 time to create 1 rle with old method : 0.022656679153442383 length of segment : 248 time for calcul the mask position with numpy : 0.0013666152954101562 nb_pixel_total : 53198 time to create 1 rle with old method : 0.06585836410522461 length of segment : 267 time for calcul the mask position with numpy : 0.0009882450103759766 nb_pixel_total : 19020 time to create 1 rle with old method : 0.02251458168029785 length of segment : 145 time for calcul the mask position with numpy : 0.0006148815155029297 nb_pixel_total : 11878 time to create 1 rle with old method : 0.01497650146484375 length of segment : 179 time for calcul the mask position with numpy : 0.00011467933654785156 nb_pixel_total : 1141 time to create 1 rle with old method : 0.0016646385192871094 length of segment : 45 time for calcul the mask position with numpy : 0.0013873577117919922 nb_pixel_total : 39847 time to create 1 rle with old method : 0.048575401306152344 length of segment : 297 time for calcul the mask position with numpy : 0.002948284149169922 nb_pixel_total : 117235 time to create 1 rle with old method : 0.1370408535003662 length of segment : 380 time for calcul the mask position with numpy : 0.00024700164794921875 nb_pixel_total : 2595 time to create 1 rle with old method : 0.0033812522888183594 length of segment : 74 time for calcul the mask position with numpy : 0.0015003681182861328 nb_pixel_total : 54126 time to create 1 rle with old method : 0.06691336631774902 length of segment : 314 time for calcul the mask position with numpy : 0.0007550716400146484 nb_pixel_total : 16725 time to create 1 rle with old method : 0.02047562599182129 length of segment : 187 time for calcul the mask position with numpy : 0.0002884864807128906 nb_pixel_total : 8799 time to create 1 rle with old method : 0.011064291000366211 length of segment : 92 time for calcul the mask position with numpy : 0.0018427371978759766 nb_pixel_total : 66851 time to create 1 rle with old method : 0.0824742317199707 length of segment : 418 time for calcul the mask position with numpy : 0.00205230712890625 nb_pixel_total : 86017 time to create 1 rle with old method : 0.12135195732116699 length of segment : 338 time for calcul the mask position with numpy : 0.0007948875427246094 nb_pixel_total : 26956 time to create 1 rle with old method : 0.03370857238769531 length of segment : 179 time for calcul the mask position with numpy : 0.0029413700103759766 nb_pixel_total : 97457 time to create 1 rle with old method : 0.1209878921508789 length of segment : 864 time for calcul the mask position with numpy : 0.1305696964263916 nb_pixel_total : 2675402 time to create 1 rle with new method : 0.5976495742797852 length of segment : 2891 time for calcul the mask position with numpy : 0.0012516975402832031 nb_pixel_total : 69798 time to create 1 rle with old method : 0.08536577224731445 length of segment : 369 time for calcul the mask position with numpy : 0.007699489593505859 nb_pixel_total : 502393 time to create 1 rle with new method : 0.016370534896850586 length of segment : 758 time for calcul the mask position with numpy : 0.0035767555236816406 nb_pixel_total : 185802 time to create 1 rle with new method : 0.008821249008178711 length of segment : 1005 time for calcul the mask position with numpy : 0.0028181076049804688 nb_pixel_total : 169198 time to create 1 rle with new method : 0.007937192916870117 length of segment : 635 time for calcul the mask position with numpy : 0.0001270771026611328 nb_pixel_total : 2717 time to create 1 rle with old method : 0.0035703182220458984 length of segment : 125 time for calcul the mask position with numpy : 0.002682924270629883 nb_pixel_total : 174742 time to create 1 rle with new method : 0.0075016021728515625 length of segment : 631 time for calcul the mask position with numpy : 0.0027360916137695312 nb_pixel_total : 168487 time to create 1 rle with new method : 0.007604837417602539 length of segment : 1053 time for calcul the mask position with numpy : 0.0015954971313476562 nb_pixel_total : 109134 time to create 1 rle with old method : 0.13368582725524902 length of segment : 493 time for calcul the mask position with numpy : 0.00025153160095214844 nb_pixel_total : 7688 time to create 1 rle with old method : 0.010101556777954102 length of segment : 159 time for calcul the mask position with numpy : 0.017832040786743164 nb_pixel_total : 877524 time to create 1 rle with new method : 0.0604245662689209 length of segment : 2216 time for calcul the mask position with numpy : 0.00031876564025878906 nb_pixel_total : 17648 time to create 1 rle with old method : 0.021249771118164062 length of segment : 180 time for calcul the mask position with numpy : 0.0002372264862060547 nb_pixel_total : 12991 time to create 1 rle with old method : 0.016849517822265625 length of segment : 149 time for calcul the mask position with numpy : 0.0020275115966796875 nb_pixel_total : 65875 time to create 1 rle with old method : 0.07956099510192871 length of segment : 342 time for calcul the mask position with numpy : 0.2644233703613281 nb_pixel_total : 2872081 time to create 1 rle with new method : 0.3969569206237793 length of segment : 2800 time for calcul the mask position with numpy : 0.00010514259338378906 nb_pixel_total : 1699 time to create 1 rle with old method : 0.0022072792053222656 length of segment : 83 time for calcul the mask position with numpy : 0.00035834312438964844 nb_pixel_total : 20444 time to create 1 rle with old method : 0.025850534439086914 length of segment : 175 time for calcul the mask position with numpy : 0.015671968460083008 nb_pixel_total : 1094661 time to create 1 rle with new method : 0.043456315994262695 length of segment : 1946 time for calcul the mask position with numpy : 0.00020360946655273438 nb_pixel_total : 3441 time to create 1 rle with old method : 0.004190683364868164 length of segment : 83 time for calcul the mask position with numpy : 0.001039743423461914 nb_pixel_total : 55360 time to create 1 rle with old method : 0.06770920753479004 length of segment : 411 time for calcul the mask position with numpy : 0.0034537315368652344 nb_pixel_total : 160722 time to create 1 rle with new method : 0.006398200988769531 length of segment : 528 time for calcul the mask position with numpy : 0.0002238750457763672 nb_pixel_total : 5956 time to create 1 rle with old method : 0.007633209228515625 length of segment : 198 time for calcul the mask position with numpy : 0.0007157325744628906 nb_pixel_total : 22052 time to create 1 rle with old method : 0.026895999908447266 length of segment : 196 time for calcul the mask position with numpy : 0.000835418701171875 nb_pixel_total : 33245 time to create 1 rle with old method : 0.04083728790283203 length of segment : 320 time for calcul the mask position with numpy : 0.0014634132385253906 nb_pixel_total : 69579 time to create 1 rle with old method : 0.0840156078338623 length of segment : 520 time for calcul the mask position with numpy : 0.0024940967559814453 nb_pixel_total : 140286 time to create 1 rle with old method : 0.1830441951751709 length of segment : 556 time for calcul the mask position with numpy : 0.0016732215881347656 nb_pixel_total : 53630 time to create 1 rle with old method : 0.06054520606994629 length of segment : 340 time for calcul the mask position with numpy : 0.004912853240966797 nb_pixel_total : 256533 time to create 1 rle with new method : 0.010681390762329102 length of segment : 954 time for calcul the mask position with numpy : 0.0066928863525390625 nb_pixel_total : 383279 time to create 1 rle with new method : 0.014496088027954102 length of segment : 916 time for calcul the mask position with numpy : 0.001920461654663086 nb_pixel_total : 76242 time to create 1 rle with old method : 0.08246278762817383 length of segment : 383 time for calcul the mask position with numpy : 0.00032830238342285156 nb_pixel_total : 14431 time to create 1 rle with old method : 0.023546934127807617 length of segment : 180 time for calcul the mask position with numpy : 0.00036215782165527344 nb_pixel_total : 25672 time to create 1 rle with old method : 0.033516883850097656 length of segment : 188 time for calcul the mask position with numpy : 0.0011243820190429688 nb_pixel_total : 57016 time to create 1 rle with old method : 0.06414151191711426 length of segment : 266 time for calcul the mask position with numpy : 0.0006885528564453125 nb_pixel_total : 47458 time to create 1 rle with old method : 0.05388307571411133 length of segment : 261 time for calcul the mask position with numpy : 0.0019106864929199219 nb_pixel_total : 54817 time to create 1 rle with old method : 0.0648496150970459 length of segment : 307 time for calcul the mask position with numpy : 0.000469207763671875 nb_pixel_total : 5945 time to create 1 rle with old method : 0.00729060173034668 length of segment : 192 time for calcul the mask position with numpy : 0.0023179054260253906 nb_pixel_total : 74004 time to create 1 rle with old method : 0.08809638023376465 length of segment : 417 time for calcul the mask position with numpy : 0.0008039474487304688 nb_pixel_total : 16812 time to create 1 rle with old method : 0.020475149154663086 length of segment : 188 time for calcul the mask position with numpy : 0.0008645057678222656 nb_pixel_total : 13561 time to create 1 rle with old method : 0.018309354782104492 length of segment : 158 time for calcul the mask position with numpy : 0.0006160736083984375 nb_pixel_total : 12351 time to create 1 rle with old method : 0.014631271362304688 length of segment : 132 time for calcul the mask position with numpy : 0.015441417694091797 nb_pixel_total : 498420 time to create 1 rle with new method : 0.020903825759887695 length of segment : 1171 time for calcul the mask position with numpy : 0.00046753883361816406 nb_pixel_total : 7849 time to create 1 rle with old method : 0.00995016098022461 length of segment : 156 time for calcul the mask position with numpy : 0.0010097026824951172 nb_pixel_total : 18398 time to create 1 rle with old method : 0.022602558135986328 length of segment : 292 time for calcul the mask position with numpy : 0.0004749298095703125 nb_pixel_total : 8661 time to create 1 rle with old method : 0.010505199432373047 length of segment : 141 time for calcul the mask position with numpy : 0.01050567626953125 nb_pixel_total : 328955 time to create 1 rle with new method : 0.013546466827392578 length of segment : 815 time for calcul the mask position with numpy : 0.0007512569427490234 nb_pixel_total : 12139 time to create 1 rle with old method : 0.014673948287963867 length of segment : 148 time for calcul the mask position with numpy : 0.006773471832275391 nb_pixel_total : 157039 time to create 1 rle with new method : 0.009692668914794922 length of segment : 772 time for calcul the mask position with numpy : 0.0014662742614746094 nb_pixel_total : 21326 time to create 1 rle with old method : 0.02432107925415039 length of segment : 533 time for calcul the mask position with numpy : 0.009884357452392578 nb_pixel_total : 361153 time to create 1 rle with new method : 0.017503738403320312 length of segment : 1532 time for calcul the mask position with numpy : 0.0008168220520019531 nb_pixel_total : 11929 time to create 1 rle with old method : 0.014599800109863281 length of segment : 302 time for calcul the mask position with numpy : 0.0017981529235839844 nb_pixel_total : 50595 time to create 1 rle with old method : 0.06875252723693848 length of segment : 236 time for calcul the mask position with numpy : 0.002482175827026367 nb_pixel_total : 114094 time to create 1 rle with old method : 0.13040637969970703 length of segment : 833 time for calcul the mask position with numpy : 0.0012733936309814453 nb_pixel_total : 29584 time to create 1 rle with old method : 0.03441762924194336 length of segment : 370 time for calcul the mask position with numpy : 0.004477024078369141 nb_pixel_total : 154650 time to create 1 rle with new method : 0.00722050666809082 length of segment : 708 time for calcul the mask position with numpy : 0.006005048751831055 nb_pixel_total : 321232 time to create 1 rle with new method : 0.011584281921386719 length of segment : 1089 time for calcul the mask position with numpy : 0.0033655166625976562 nb_pixel_total : 140269 time to create 1 rle with old method : 0.2156980037689209 length of segment : 559 time for calcul the mask position with numpy : 0.0008306503295898438 nb_pixel_total : 16852 time to create 1 rle with old method : 0.019963741302490234 length of segment : 168 time for calcul the mask position with numpy : 0.003014087677001953 nb_pixel_total : 162494 time to create 1 rle with new method : 0.0080413818359375 length of segment : 632 time for calcul the mask position with numpy : 0.0006120204925537109 nb_pixel_total : 31012 time to create 1 rle with old method : 0.03878164291381836 length of segment : 189 time for calcul the mask position with numpy : 0.0012443065643310547 nb_pixel_total : 29122 time to create 1 rle with old method : 0.03551840782165527 length of segment : 304 time for calcul the mask position with numpy : 0.013274431228637695 nb_pixel_total : 544737 time to create 1 rle with new method : 0.024179697036743164 length of segment : 1216 time for calcul the mask position with numpy : 0.0008361339569091797 nb_pixel_total : 10810 time to create 1 rle with old method : 0.02994680404663086 length of segment : 167 time for calcul the mask position with numpy : 0.0004787445068359375 nb_pixel_total : 9456 time to create 1 rle with old method : 0.011957168579101562 length of segment : 107 time for calcul the mask position with numpy : 0.0007140636444091797 nb_pixel_total : 8233 time to create 1 rle with old method : 0.011183977127075195 length of segment : 101 time for calcul the mask position with numpy : 0.0070590972900390625 nb_pixel_total : 158577 time to create 1 rle with new method : 0.011040210723876953 length of segment : 619 time for calcul the mask position with numpy : 0.0010287761688232422 nb_pixel_total : 33115 time to create 1 rle with old method : 0.0665593147277832 length of segment : 215 time for calcul the mask position with numpy : 0.0004763603210449219 nb_pixel_total : 11417 time to create 1 rle with old method : 0.013911247253417969 length of segment : 124 time for calcul the mask position with numpy : 0.0002448558807373047 nb_pixel_total : 3845 time to create 1 rle with old method : 0.00481104850769043 length of segment : 78 time for calcul the mask position with numpy : 0.0008697509765625 nb_pixel_total : 19572 time to create 1 rle with old method : 0.024651527404785156 length of segment : 194 time for calcul the mask position with numpy : 0.00030303001403808594 nb_pixel_total : 5644 time to create 1 rle with old method : 0.007264852523803711 length of segment : 141 time for calcul the mask position with numpy : 0.006803035736083984 nb_pixel_total : 215363 time to create 1 rle with new method : 0.008342742919921875 length of segment : 539 time for calcul the mask position with numpy : 0.0031480789184570312 nb_pixel_total : 97166 time to create 1 rle with old method : 0.11238718032836914 length of segment : 559 time for calcul the mask position with numpy : 0.004591226577758789 nb_pixel_total : 152694 time to create 1 rle with new method : 0.006952047348022461 length of segment : 626 time for calcul the mask position with numpy : 0.0003368854522705078 nb_pixel_total : 9819 time to create 1 rle with old method : 0.01287388801574707 length of segment : 85 time for calcul the mask position with numpy : 0.001211404800415039 nb_pixel_total : 21234 time to create 1 rle with old method : 0.03395485877990723 length of segment : 257 time for calcul the mask position with numpy : 0.0010752677917480469 nb_pixel_total : 14628 time to create 1 rle with old method : 0.025696754455566406 length of segment : 139 time for calcul the mask position with numpy : 0.0005061626434326172 nb_pixel_total : 22723 time to create 1 rle with old method : 0.0282745361328125 length of segment : 139 time for calcul the mask position with numpy : 0.0017566680908203125 nb_pixel_total : 84721 time to create 1 rle with old method : 0.0960843563079834 length of segment : 360 time for calcul the mask position with numpy : 0.0006604194641113281 nb_pixel_total : 24929 time to create 1 rle with old method : 0.029385089874267578 length of segment : 97 time for calcul the mask position with numpy : 0.0017631053924560547 nb_pixel_total : 33530 time to create 1 rle with old method : 0.039020538330078125 length of segment : 467 time for calcul the mask position with numpy : 0.0003228187561035156 nb_pixel_total : 14857 time to create 1 rle with old method : 0.017443180084228516 length of segment : 166 time for calcul the mask position with numpy : 0.0036208629608154297 nb_pixel_total : 111148 time to create 1 rle with old method : 0.12553930282592773 length of segment : 380 time for calcul the mask position with numpy : 0.0002434253692626953 nb_pixel_total : 5364 time to create 1 rle with old method : 0.007081508636474609 length of segment : 84 time for calcul the mask position with numpy : 0.0018582344055175781 nb_pixel_total : 69123 time to create 1 rle with old method : 0.08062148094177246 length of segment : 305 time for calcul the mask position with numpy : 0.0006780624389648438 nb_pixel_total : 14549 time to create 1 rle with old method : 0.01840686798095703 length of segment : 132 time for calcul the mask position with numpy : 0.00014519691467285156 nb_pixel_total : 2475 time to create 1 rle with old method : 0.003895998001098633 length of segment : 63 time for calcul the mask position with numpy : 0.0028543472290039062 nb_pixel_total : 159682 time to create 1 rle with new method : 0.007951021194458008 length of segment : 681 time for calcul the mask position with numpy : 0.0003237724304199219 nb_pixel_total : 6825 time to create 1 rle with old method : 0.008659839630126953 length of segment : 108 time for calcul the mask position with numpy : 0.005223274230957031 nb_pixel_total : 270057 time to create 1 rle with new method : 0.01257634162902832 length of segment : 700 time spent for convertir_results : 21.642253637313843 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 186 chid ids of type : 3663 Number RLEs to save : 72887 save missing photos in datou_result : time spend for datou_step_exec : 78.22322225570679 time spend to save output : 4.34297513961792 total time spend for step 1 : 82.5661973953247 step2:crop_condition Thu Feb 27 16:03:51 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 186 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 28 About to insert : list_path_to_insert length 28 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 18 About to insert : list_path_to_insert length 18 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 55 About to insert : list_path_to_insert length 55 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 9 About to insert : list_path_to_insert length 9 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1339768163, 1339768140, 1339768138, 1339768132, 1339768127, 1339767994, 1339767992, 1339767989, 1339767985, 1339767964] Looping around the photos to save general results len do output : 121 /-3688784386Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784391Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784387Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784390Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784402Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784397Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784396Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784425Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784417Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784443Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784440Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784439Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784458Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784460Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784457Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784468Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784478Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784487Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784488Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784496Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784505Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784522Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784528Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784521Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784517Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784554Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784547Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784567Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784405Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784411Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784398Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784423Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784434Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784431Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784428Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784436Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784454Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784465Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784467Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784480Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784477Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784476Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784511Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784502Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784560Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784558Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784466Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784512Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784503Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784540Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784555Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784393Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784394Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784408Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784410Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784409Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784399Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784416Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784406Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784429Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784421Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784422Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784420Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784432Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784437Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784427Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784446Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784448Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784444Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784442Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784449Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784451Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784461Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784463Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784481Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784482Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784469Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784474Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784479Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784473Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784494Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784489Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784491Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784490Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784507Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784509Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784515Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784506Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784513Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784508Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784519Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784542Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784532Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784529Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784535Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784533Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784536Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784526Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784520Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784537Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784571Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784564Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784563Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784569Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784562Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784557Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784392Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784407Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784424Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784418Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784464Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784499Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784561Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784551Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784559Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784400Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784426Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784419Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784441Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784484Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688784566Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768163', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768140', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768138', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768132', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768127', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767994', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767992', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767989', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767985', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767964', None, None, None, None, None, '2619767') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 373 time used for this insertion : 0.03989672660827637 save_final save missing photos in datou_result : time spend for datou_step_exec : 43.90929698944092 time spend to save output : 0.04273581504821777 total time spend for step 2 : 43.952032804489136 step3:thcl Thu Feb 27 16:04:35 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 2 we are using the classfication for multi_thcl [2456, 2868] time to import caffe and check if the image exist : 0.014461517333984375 time to convert the images to numpy array : 0.08185124397277832 time to import caffe and check if the image exist : 0.02443695068359375 time to convert the images to numpy array : 0.09330391883850098 time to import caffe and check if the image exist : 0.023863792419433594 time to convert the images to numpy array : 0.10429739952087402 time to import caffe and check if the image exist : 0.01795053482055664 time to convert the images to numpy array : 0.14643621444702148 time to import caffe and check if the image exist : 0.020305871963500977 time to convert the images to numpy array : 0.1612389087677002 time to import caffe and check if the image exist : 0.022697925567626953 time to convert the images to numpy array : 0.1666245460510254 time to import caffe and check if the image exist : 0.02219223976135254 time to convert the images to numpy array : 0.18263602256774902 time to import caffe and check if the image exist : 0.018576383590698242 time to convert the images to numpy array : 0.19499588012695312 time to import caffe and check if the image exist : 0.02391505241394043 time to convert the images to numpy array : 0.26849842071533203 time to import caffe and check if the image exist : 0.021328210830688477 time to convert the images to numpy array : 0.2900822162628174 total time to convert the images to numpy array : 0.8625633716583252 list photo_ids error: [] list photo_ids correct : [-3688784419, -3688784441, -3688784484, -3688784566, -3688784562, -3688784557, -3688784392, -3688784407, -3688784424, -3688784418, -3688784464, -3688784499, -3688784561, -3688784551, -3688784559, -3688784400, -3688784426, -3688784460, -3688784457, -3688784468, -3688784478, -3688784487, -3688784488, -3688784496, -3688784505, -3688784522, -3688784528, -3688784521, -3688784517, -3688784554, -3688784386, -3688784391, -3688784387, -3688784390, -3688784402, -3688784397, -3688784396, -3688784425, -3688784417, -3688784443, -3688784440, -3688784439, -3688784458, -3688784394, -3688784408, -3688784410, -3688784409, -3688784399, -3688784416, -3688784406, -3688784429, -3688784421, -3688784422, -3688784420, -3688784432, -3688784437, -3688784547, -3688784567, -3688784405, -3688784411, -3688784398, -3688784423, -3688784434, -3688784431, -3688784428, -3688784436, -3688784454, -3688784465, -3688784467, -3688784427, -3688784446, -3688784448, -3688784444, -3688784442, -3688784449, -3688784451, -3688784461, -3688784463, -3688784481, -3688784482, -3688784469, -3688784474, -3688784479, -3688784473, -3688784494, -3688784489, -3688784491, -3688784490, -3688784507, -3688784509, -3688784515, -3688784506, -3688784513, -3688784508, -3688784519, -3688784542, -3688784532, -3688784529, -3688784535, -3688784533, -3688784536, -3688784526, -3688784520, -3688784537, -3688784571, -3688784564, -3688784563, -3688784569, -3688784480, -3688784477, -3688784476, -3688784511, -3688784502, -3688784560, -3688784558, -3688784466, -3688784512, -3688784503, -3688784540, -3688784555, -3688784393] number of photos to traite : 121 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10814 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10595 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.2326209545135498 time used to do the prediction : 0.3122138977050781 we don't save the descriptors for this thcl 2456 tagging for thcl : 2868 To do loadFromThcl(), then load ParamDescType : thcl2868 thcls : [{'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5288 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) To loadFromThcl() : net_5288 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10593 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_papier_nantes_300421 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_papier_nantes_300421 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_papier_nantes_300421 /data/models_weight/learn_papier_nantes_300421/caffemodel size_local : 44791983 size in s3 : 44791983 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:13 caffemodel already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/deploy.prototxt size_local : 17255 size in s3 : 17255 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:12 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:14 mean.npy already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/synset_words.txt size_local : 331 size in s3 : 331 create time local : 2021-08-09 05:56:00 create time in s3 : 2021-08-06 19:22:12 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_papier_nantes_300421/deploy.prototxt caffemodel_filename : /data/models_weight/learn_papier_nantes_300421/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10593 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.2583260536193848 time used to do the prediction : 0.22365117073059082 we don't save the descriptors for this thcl 2868 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 3.0994415283203125e-05 save missing photos in datou_result : time spend for datou_step_exec : 11.155736684799194 time spend to save output : 0.0013213157653808594 total time spend for step 3 : 11.157058000564575 step4:merge_mask_thcl_custom Thu Feb 27 16:04:46 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 186 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1339768163, 1339768140, 1339768138, 1339768132, 1339768127, 1339767994, 1339767992, 1339767989, 1339767985, 1339767964] Looping around the photos to save general results len do output : 10 /1339768163Didn't retrieve data .Didn't retrieve data . /1339768140Didn't retrieve data .Didn't retrieve data . /1339768138Didn't retrieve data .Didn't retrieve data . /1339768132Didn't retrieve data .Didn't retrieve data . /1339768127Didn't retrieve data .Didn't retrieve data . /1339767994Didn't retrieve data .Didn't retrieve data . /1339767992Didn't retrieve data .Didn't retrieve data . /1339767989Didn't retrieve data .Didn't retrieve data . /1339767985Didn't retrieve data .Didn't retrieve data . /1339767964Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768163', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768140', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768138', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768132', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768127', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767994', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767992', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767989', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767985', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767964', None, None, None, None, None, '2619767') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.014279842376708984 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.06451606750488281 time spend to save output : 0.014744281768798828 total time spend for step 4 : 0.07926034927368164 step5:rle_unique_nms_with_priority Thu Feb 27 16:04:46 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 10 nb_hashtags : 4 time to prepare the origin masks : 5.394065618515015 time for calcul the mask position with numpy : 0.3166472911834717 nb_pixel_total : 4322604 time to create 1 rle with new method : 0.8734242916107178 time for calcul the mask position with numpy : 0.03445267677307129 nb_pixel_total : 214852 time to create 1 rle with new method : 0.3723626136779785 time for calcul the mask position with numpy : 0.02275848388671875 nb_pixel_total : 7174 time to create 1 rle with old method : 0.008543968200683594 time for calcul the mask position with numpy : 0.02134990692138672 nb_pixel_total : 14339 time to create 1 rle with old method : 0.015959978103637695 time for calcul the mask position with numpy : 0.022013187408447266 nb_pixel_total : 77534 time to create 1 rle with old method : 0.08429741859436035 time for calcul the mask position with numpy : 0.020237207412719727 nb_pixel_total : 40442 time to create 1 rle with old method : 0.04368090629577637 time for calcul the mask position with numpy : 0.0339818000793457 nb_pixel_total : 161456 time to create 1 rle with new method : 0.5004944801330566 time for calcul the mask position with numpy : 0.0436704158782959 nb_pixel_total : 588585 time to create 1 rle with new method : 0.3777134418487549 time for calcul the mask position with numpy : 0.09732770919799805 nb_pixel_total : 1356222 time to create 1 rle with new method : 0.8004631996154785 time for calcul the mask position with numpy : 0.036147356033325195 nb_pixel_total : 240468 time to create 1 rle with new method : 0.3933858871459961 time for calcul the mask position with numpy : 0.0328977108001709 nb_pixel_total : 26564 time to create 1 rle with old method : 0.029499530792236328 create new chi : 4.326869487762451 time to delete rle : 0.019942522048950195 batch 1 Loaded 11 chid ids of type : 3726 Number RLEs to save : 13771 TO DO : save crop sub photo not yet done ! save time : 1.2709853649139404 nb_obj : 21 nb_hashtags : 7 time to prepare the origin masks : 13.617931365966797 time for calcul the mask position with numpy : 0.4257314205169678 nb_pixel_total : 6463276 time to create 1 rle with new method : 0.8778371810913086 time for calcul the mask position with numpy : 0.037513017654418945 nb_pixel_total : 58087 time to create 1 rle with old method : 0.06362271308898926 time for calcul the mask position with numpy : 0.0330808162689209 nb_pixel_total : 9144 time to create 1 rle with old method : 0.010883569717407227 time for calcul the mask position with numpy : 0.03451251983642578 nb_pixel_total : 29642 time to create 1 rle with old method : 0.03551626205444336 time for calcul the mask position with numpy : 0.03517007827758789 nb_pixel_total : 60012 time to create 1 rle with old method : 0.06619906425476074 time for calcul the mask position with numpy : 0.033476829528808594 nb_pixel_total : 60979 time to create 1 rle with old method : 0.07191658020019531 time for calcul the mask position with numpy : 0.031807661056518555 nb_pixel_total : 15435 time to create 1 rle with old method : 0.017520904541015625 time for calcul the mask position with numpy : 0.03459429740905762 nb_pixel_total : 11260 time to create 1 rle with old method : 0.013135194778442383 time for calcul the mask position with numpy : 0.03392291069030762 nb_pixel_total : 2637 time to create 1 rle with old method : 0.0029904842376708984 time for calcul the mask position with numpy : 0.034918785095214844 nb_pixel_total : 5069 time to create 1 rle with old method : 0.006045341491699219 time for calcul the mask position with numpy : 0.03651928901672363 nb_pixel_total : 7389 time to create 1 rle with old method : 0.01445913314819336 time for calcul the mask position with numpy : 0.044315338134765625 nb_pixel_total : 59328 time to create 1 rle with old method : 0.06672883033752441 time for calcul the mask position with numpy : 0.03577232360839844 nb_pixel_total : 25367 time to create 1 rle with old method : 0.04120469093322754 time for calcul the mask position with numpy : 0.022549867630004883 nb_pixel_total : 7226 time to create 1 rle with old method : 0.010236501693725586 time for calcul the mask position with numpy : 0.021620988845825195 nb_pixel_total : 2662 time to create 1 rle with old method : 0.0031888484954833984 time for calcul the mask position with numpy : 0.021599531173706055 nb_pixel_total : 1498 time to create 1 rle with old method : 0.0018126964569091797 time for calcul the mask position with numpy : 0.02207159996032715 nb_pixel_total : 43519 time to create 1 rle with old method : 0.05960869789123535 time for calcul the mask position with numpy : 0.024267911911010742 nb_pixel_total : 5410 time to create 1 rle with old method : 0.010736227035522461 time for calcul the mask position with numpy : 0.025182247161865234 nb_pixel_total : 99546 time to create 1 rle with old method : 0.14479804039001465 time for calcul the mask position with numpy : 0.043968915939331055 nb_pixel_total : 27059 time to create 1 rle with old method : 0.03946709632873535 time for calcul the mask position with numpy : 0.03474617004394531 nb_pixel_total : 28547 time to create 1 rle with old method : 0.03273725509643555 time for calcul the mask position with numpy : 0.036359310150146484 nb_pixel_total : 27148 time to create 1 rle with old method : 0.029523611068725586 create new chi : 2.768385171890259 time to delete rle : 0.00122833251953125 batch 1 Loaded 22 chid ids of type : 3726 Number RLEs to save : 12036 TO DO : save crop sub photo not yet done ! save time : 0.984184741973877 nb_obj : 21 nb_hashtags : 8 time to prepare the origin masks : 13.643810033798218 time for calcul the mask position with numpy : 0.33715009689331055 nb_pixel_total : 5996492 time to create 1 rle with new method : 0.6118409633636475 time for calcul the mask position with numpy : 0.035086631774902344 nb_pixel_total : 313 time to create 1 rle with old method : 0.00047516822814941406 time for calcul the mask position with numpy : 0.034967660903930664 nb_pixel_total : 56218 time to create 1 rle with old method : 0.062131643295288086 time for calcul the mask position with numpy : 0.0353546142578125 nb_pixel_total : 44906 time to create 1 rle with old method : 0.07286286354064941 time for calcul the mask position with numpy : 0.03619885444641113 nb_pixel_total : 75243 time to create 1 rle with old method : 0.08321714401245117 time for calcul the mask position with numpy : 0.03621077537536621 nb_pixel_total : 29736 time to create 1 rle with old method : 0.034249067306518555 time for calcul the mask position with numpy : 0.03420305252075195 nb_pixel_total : 3951 time to create 1 rle with old method : 0.004297971725463867 time for calcul the mask position with numpy : 0.03420138359069824 nb_pixel_total : 1247 time to create 1 rle with old method : 0.00144195556640625 time for calcul the mask position with numpy : 0.03411126136779785 nb_pixel_total : 39031 time to create 1 rle with old method : 0.04663395881652832 time for calcul the mask position with numpy : 0.03367424011230469 nb_pixel_total : 65640 time to create 1 rle with old method : 0.08824634552001953 time for calcul the mask position with numpy : 0.034208059310913086 nb_pixel_total : 18359 time to create 1 rle with old method : 0.019950151443481445 time for calcul the mask position with numpy : 0.03377270698547363 nb_pixel_total : 9708 time to create 1 rle with old method : 0.010648012161254883 time for calcul the mask position with numpy : 0.03396749496459961 nb_pixel_total : 21614 time to create 1 rle with old method : 0.023801803588867188 time for calcul the mask position with numpy : 0.03349947929382324 nb_pixel_total : 72268 time to create 1 rle with old method : 0.08379340171813965 time for calcul the mask position with numpy : 0.0339813232421875 nb_pixel_total : 61316 time to create 1 rle with old method : 0.07408952713012695 time for calcul the mask position with numpy : 0.042250871658325195 nb_pixel_total : 27371 time to create 1 rle with old method : 0.0368494987487793 time for calcul the mask position with numpy : 0.038842201232910156 nb_pixel_total : 49567 time to create 1 rle with old method : 0.059061288833618164 time for calcul the mask position with numpy : 0.037343740463256836 nb_pixel_total : 216606 time to create 1 rle with new method : 0.4889037609100342 time for calcul the mask position with numpy : 0.03090524673461914 nb_pixel_total : 42610 time to create 1 rle with old method : 0.04584813117980957 time for calcul the mask position with numpy : 0.02737903594970703 nb_pixel_total : 10651 time to create 1 rle with old method : 0.01233530044555664 time for calcul the mask position with numpy : 0.03130483627319336 nb_pixel_total : 168797 time to create 1 rle with new method : 0.3996748924255371 time for calcul the mask position with numpy : 0.02208709716796875 nb_pixel_total : 38596 time to create 1 rle with old method : 0.044506072998046875 create new chi : 3.4452731609344482 time to delete rle : 0.001939535140991211 batch 1 Loaded 22 chid ids of type : 3726 Number RLEs to save : 15247 TO DO : save crop sub photo not yet done ! save time : 0.8719925880432129 nb_obj : 16 nb_hashtags : 6 time to prepare the origin masks : 9.892037868499756 time for calcul the mask position with numpy : 0.3039109706878662 nb_pixel_total : 6535636 time to create 1 rle with new method : 0.3201329708099365 time for calcul the mask position with numpy : 0.03544116020202637 nb_pixel_total : 41858 time to create 1 rle with old method : 0.04735398292541504 time for calcul the mask position with numpy : 0.034784555435180664 nb_pixel_total : 57956 time to create 1 rle with old method : 0.07338786125183105 time for calcul the mask position with numpy : 0.042883872985839844 nb_pixel_total : 29775 time to create 1 rle with old method : 0.03444933891296387 time for calcul the mask position with numpy : 0.03382062911987305 nb_pixel_total : 9084 time to create 1 rle with old method : 0.011114120483398438 time for calcul the mask position with numpy : 0.03427600860595703 nb_pixel_total : 1329 time to create 1 rle with old method : 0.0015482902526855469 time for calcul the mask position with numpy : 0.02356863021850586 nb_pixel_total : 61186 time to create 1 rle with old method : 0.0682377815246582 time for calcul the mask position with numpy : 0.02480936050415039 nb_pixel_total : 55672 time to create 1 rle with old method : 0.06391334533691406 time for calcul the mask position with numpy : 0.021314620971679688 nb_pixel_total : 2251 time to create 1 rle with old method : 0.002543926239013672 time for calcul the mask position with numpy : 0.02116990089416504 nb_pixel_total : 5146 time to create 1 rle with old method : 0.006025075912475586 time for calcul the mask position with numpy : 0.021914005279541016 nb_pixel_total : 107888 time to create 1 rle with old method : 0.12071752548217773 time for calcul the mask position with numpy : 0.02248525619506836 nb_pixel_total : 4938 time to create 1 rle with old method : 0.0055696964263916016 time for calcul the mask position with numpy : 0.022038698196411133 nb_pixel_total : 46101 time to create 1 rle with old method : 0.05053091049194336 time for calcul the mask position with numpy : 0.022052288055419922 nb_pixel_total : 2521 time to create 1 rle with old method : 0.0029180049896240234 time for calcul the mask position with numpy : 0.033184051513671875 nb_pixel_total : 29765 time to create 1 rle with old method : 0.03298640251159668 time for calcul the mask position with numpy : 0.03403425216674805 nb_pixel_total : 29171 time to create 1 rle with old method : 0.0313873291015625 time for calcul the mask position with numpy : 0.03394031524658203 nb_pixel_total : 29963 time to create 1 rle with old method : 0.03239941596984863 create new chi : 1.708207130432129 time to delete rle : 0.0009596347808837891 batch 1 Loaded 17 chid ids of type : 3726 Number RLEs to save : 10023 TO DO : save crop sub photo not yet done ! save time : 1.2663905620574951 nb_obj : 12 nb_hashtags : 4 time to prepare the origin masks : 7.163474082946777 time for calcul the mask position with numpy : 0.4892466068267822 nb_pixel_total : 6545478 time to create 1 rle with new method : 0.35476160049438477 time for calcul the mask position with numpy : 0.029888391494750977 nb_pixel_total : 36637 time to create 1 rle with old method : 0.05336737632751465 time for calcul the mask position with numpy : 0.02161717414855957 nb_pixel_total : 48257 time to create 1 rle with old method : 0.05545401573181152 time for calcul the mask position with numpy : 0.04006242752075195 nb_pixel_total : 117884 time to create 1 rle with old method : 0.14243555068969727 time for calcul the mask position with numpy : 0.03548550605773926 nb_pixel_total : 3356 time to create 1 rle with old method : 0.00396418571472168 time for calcul the mask position with numpy : 0.03600597381591797 nb_pixel_total : 5690 time to create 1 rle with old method : 0.006867647171020508 time for calcul the mask position with numpy : 0.03558802604675293 nb_pixel_total : 18226 time to create 1 rle with old method : 0.020004987716674805 time for calcul the mask position with numpy : 0.03429222106933594 nb_pixel_total : 14814 time to create 1 rle with old method : 0.016248703002929688 time for calcul the mask position with numpy : 0.033577919006347656 nb_pixel_total : 89479 time to create 1 rle with old method : 0.10528945922851562 time for calcul the mask position with numpy : 0.031005859375 nb_pixel_total : 20080 time to create 1 rle with old method : 0.02229142189025879 time for calcul the mask position with numpy : 0.03273272514343262 nb_pixel_total : 26344 time to create 1 rle with old method : 0.02925848960876465 time for calcul the mask position with numpy : 0.02235579490661621 nb_pixel_total : 57975 time to create 1 rle with old method : 0.063690185546875 time for calcul the mask position with numpy : 0.022377729415893555 nb_pixel_total : 66020 time to create 1 rle with old method : 0.07622361183166504 create new chi : 1.8521618843078613 time to delete rle : 0.00078582763671875 batch 1 Loaded 13 chid ids of type : 3726 Number RLEs to save : 7550 TO DO : save crop sub photo not yet done ! save time : 0.48545241355895996 nb_obj : 14 nb_hashtags : 5 time to prepare the origin masks : 13.731380224227905 time for calcul the mask position with numpy : 0.44416165351867676 nb_pixel_total : 6452502 time to create 1 rle with new method : 0.5146164894104004 time for calcul the mask position with numpy : 0.0341334342956543 nb_pixel_total : 16670 time to create 1 rle with old method : 0.018279075622558594 time for calcul the mask position with numpy : 0.03414034843444824 nb_pixel_total : 117154 time to create 1 rle with old method : 0.13010239601135254 time for calcul the mask position with numpy : 0.03458046913146973 nb_pixel_total : 53969 time to create 1 rle with old method : 0.08909249305725098 time for calcul the mask position with numpy : 0.03860759735107422 nb_pixel_total : 8741 time to create 1 rle with old method : 0.009860992431640625 time for calcul the mask position with numpy : 0.03443789482116699 nb_pixel_total : 25152 time to create 1 rle with old method : 0.028989076614379883 time for calcul the mask position with numpy : 0.03507399559020996 nb_pixel_total : 2583 time to create 1 rle with old method : 0.0029265880584716797 time for calcul the mask position with numpy : 0.03416275978088379 nb_pixel_total : 96234 time to create 1 rle with old method : 0.12946867942810059 time for calcul the mask position with numpy : 0.04855918884277344 nb_pixel_total : 39767 time to create 1 rle with old method : 0.07064414024353027 time for calcul the mask position with numpy : 0.04446840286254883 nb_pixel_total : 85757 time to create 1 rle with old method : 0.09777021408081055 time for calcul the mask position with numpy : 0.03463006019592285 nb_pixel_total : 1138 time to create 1 rle with old method : 0.0013110637664794922 time for calcul the mask position with numpy : 0.035224199295043945 nb_pixel_total : 66738 time to create 1 rle with old method : 0.10293054580688477 time for calcul the mask position with numpy : 0.034738779067993164 nb_pixel_total : 53134 time to create 1 rle with old method : 0.05743241310119629 time for calcul the mask position with numpy : 0.03431510925292969 nb_pixel_total : 11831 time to create 1 rle with old method : 0.013114213943481445 time for calcul the mask position with numpy : 0.034879446029663086 nb_pixel_total : 18870 time to create 1 rle with old method : 0.02108144760131836 create new chi : 2.283454179763794 time to delete rle : 0.0009551048278808594 batch 1 Loaded 15 chid ids of type : 3726 Number RLEs to save : 9397 TO DO : save crop sub photo not yet done ! save time : 0.6177358627319336 nb_obj : 13 nb_hashtags : 5 time to prepare the origin masks : 12.05344820022583 time for calcul the mask position with numpy : 0.16678118705749512 nb_pixel_total : 2981126 time to create 1 rle with new method : 0.5014042854309082 time for calcul the mask position with numpy : 0.02104473114013672 nb_pixel_total : 2864 time to create 1 rle with old method : 0.0032134056091308594 time for calcul the mask position with numpy : 0.021134138107299805 nb_pixel_total : 7792 time to create 1 rle with old method : 0.008525371551513672 time for calcul the mask position with numpy : 0.021149396896362305 nb_pixel_total : 150271 time to create 1 rle with new method : 0.6545650959014893 time for calcul the mask position with numpy : 0.029704570770263672 nb_pixel_total : 2694 time to create 1 rle with old method : 0.004045963287353516 time for calcul the mask position with numpy : 0.02603316307067871 nb_pixel_total : 7660 time to create 1 rle with old method : 0.008623838424682617 time for calcul the mask position with numpy : 0.028894424438476562 nb_pixel_total : 183967 time to create 1 rle with new method : 1.113800287246704 time for calcul the mask position with numpy : 0.027782917022705078 nb_pixel_total : 501494 time to create 1 rle with new method : 0.9364800453186035 time for calcul the mask position with numpy : 0.024929046630859375 nb_pixel_total : 1862 time to create 1 rle with old method : 0.003769397735595703 time for calcul the mask position with numpy : 0.027028560638427734 nb_pixel_total : 174210 time to create 1 rle with new method : 0.6369600296020508 time for calcul the mask position with numpy : 0.27088332176208496 nb_pixel_total : 2840097 time to create 1 rle with new method : 0.49761366844177246 time for calcul the mask position with numpy : 0.02590489387512207 nb_pixel_total : 109040 time to create 1 rle with old method : 0.12234044075012207 time for calcul the mask position with numpy : 0.0369572639465332 nb_pixel_total : 17582 time to create 1 rle with old method : 0.0196533203125 time for calcul the mask position with numpy : 0.03779149055480957 nb_pixel_total : 69581 time to create 1 rle with old method : 0.10233354568481445 create new chi : 5.546358346939087 time to delete rle : 0.002085447311401367 batch 1 Loaded 14 chid ids of type : 3726 Number RLEs to save : 15557 TO DO : save crop sub photo not yet done ! save time : 0.9167642593383789 nb_obj : 21 nb_hashtags : 8 time to prepare the origin masks : 15.475070238113403 time for calcul the mask position with numpy : 0.13609075546264648 nb_pixel_total : 2861242 time to create 1 rle with new method : 1.9498300552368164 time for calcul the mask position with numpy : 0.0331273078918457 nb_pixel_total : 162436 time to create 1 rle with new method : 1.000922441482544 time for calcul the mask position with numpy : 0.038125038146972656 nb_pixel_total : 382691 time to create 1 rle with new method : 0.7330291271209717 time for calcul the mask position with numpy : 0.04051995277404785 nb_pixel_total : 20430 time to create 1 rle with old method : 0.033460140228271484 time for calcul the mask position with numpy : 0.03873085975646973 nb_pixel_total : 53527 time to create 1 rle with old method : 0.058977603912353516 time for calcul the mask position with numpy : 0.038841962814331055 nb_pixel_total : 1142 time to create 1 rle with old method : 0.001611471176147461 time for calcul the mask position with numpy : 0.04264330863952637 nb_pixel_total : 25583 time to create 1 rle with old method : 0.028107404708862305 time for calcul the mask position with numpy : 0.03461885452270508 nb_pixel_total : 21981 time to create 1 rle with old method : 0.02533411979675293 time for calcul the mask position with numpy : 0.0385136604309082 nb_pixel_total : 945 time to create 1 rle with old method : 0.001318216323852539 time for calcul the mask position with numpy : 0.03685617446899414 nb_pixel_total : 19853 time to create 1 rle with old method : 0.025321245193481445 time for calcul the mask position with numpy : 0.03003549575805664 nb_pixel_total : 8680 time to create 1 rle with old method : 0.009762763977050781 time for calcul the mask position with numpy : 0.03563261032104492 nb_pixel_total : 76207 time to create 1 rle with old method : 0.08964014053344727 time for calcul the mask position with numpy : 0.04388928413391113 nb_pixel_total : 230913 time to create 1 rle with new method : 0.7106492519378662 time for calcul the mask position with numpy : 0.03663229942321777 nb_pixel_total : 5935 time to create 1 rle with old method : 0.008529424667358398 time for calcul the mask position with numpy : 0.037749290466308594 nb_pixel_total : 132559 time to create 1 rle with old method : 0.17812824249267578 time for calcul the mask position with numpy : 0.03795313835144043 nb_pixel_total : 3432 time to create 1 rle with old method : 0.004059553146362305 time for calcul the mask position with numpy : 0.0355534553527832 nb_pixel_total : 11879 time to create 1 rle with old method : 0.013838529586791992 time for calcul the mask position with numpy : 0.03632497787475586 nb_pixel_total : 56855 time to create 1 rle with old method : 0.07509016990661621 time for calcul the mask position with numpy : 0.04271364212036133 nb_pixel_total : 69450 time to create 1 rle with old method : 0.09781694412231445 time for calcul the mask position with numpy : 0.044992923736572266 nb_pixel_total : 1697 time to create 1 rle with old method : 0.0025081634521484375 time for calcul the mask position with numpy : 0.2212231159210205 nb_pixel_total : 2847640 time to create 1 rle with new method : 1.3621826171875 time for calcul the mask position with numpy : 0.03468179702758789 nb_pixel_total : 55163 time to create 1 rle with old method : 0.06328582763671875 create new chi : 7.724806785583496 time to delete rle : 0.0028367042541503906 batch 1 Loaded 22 chid ids of type : 3726 Number RLEs to save : 20739 TO DO : save crop sub photo not yet done ! save time : 1.246488094329834 nb_obj : 23 nb_hashtags : 7 time to prepare the origin masks : 14.30341649055481 time for calcul the mask position with numpy : 0.3448150157928467 nb_pixel_total : 4952504 time to create 1 rle with new method : 0.873340368270874 time for calcul the mask position with numpy : 0.025232791900634766 nb_pixel_total : 73829 time to create 1 rle with old method : 0.07883572578430176 time for calcul the mask position with numpy : 0.02321338653564453 nb_pixel_total : 9643 time to create 1 rle with old method : 0.010921239852905273 time for calcul the mask position with numpy : 0.022946596145629883 nb_pixel_total : 717 time to create 1 rle with old method : 0.28142333030700684 time for calcul the mask position with numpy : 0.028795242309570312 nb_pixel_total : 11910 time to create 1 rle with old method : 0.012752294540405273 time for calcul the mask position with numpy : 0.029173612594604492 nb_pixel_total : 50416 time to create 1 rle with old method : 0.05239510536193848 time for calcul the mask position with numpy : 0.03865313529968262 nb_pixel_total : 29008 time to create 1 rle with old method : 0.03162884712219238 time for calcul the mask position with numpy : 0.04258918762207031 nb_pixel_total : 240292 time to create 1 rle with new method : 0.7236216068267822 time for calcul the mask position with numpy : 0.039453983306884766 nb_pixel_total : 29395 time to create 1 rle with old method : 0.03032064437866211 time for calcul the mask position with numpy : 0.039032936096191406 nb_pixel_total : 8642 time to create 1 rle with old method : 0.00988006591796875 time for calcul the mask position with numpy : 0.0401463508605957 nb_pixel_total : 30447 time to create 1 rle with old method : 0.03245735168457031 time for calcul the mask position with numpy : 0.04652118682861328 nb_pixel_total : 21279 time to create 1 rle with old method : 0.024623870849609375 time for calcul the mask position with numpy : 0.0397944450378418 nb_pixel_total : 12321 time to create 1 rle with old method : 0.014419078826904297 time for calcul the mask position with numpy : 0.041916847229003906 nb_pixel_total : 154537 time to create 1 rle with new method : 1.517580270767212 time for calcul the mask position with numpy : 0.04180765151977539 nb_pixel_total : 328033 time to create 1 rle with new method : 0.7953722476959229 time for calcul the mask position with numpy : 0.03878068923950195 nb_pixel_total : 325447 time to create 1 rle with new method : 0.8798446655273438 time for calcul the mask position with numpy : 0.03527522087097168 nb_pixel_total : 18381 time to create 1 rle with old method : 0.020901203155517578 time for calcul the mask position with numpy : 0.03648996353149414 nb_pixel_total : 7828 time to create 1 rle with old method : 0.009531974792480469 time for calcul the mask position with numpy : 0.03902077674865723 nb_pixel_total : 498259 time to create 1 rle with new method : 0.9794301986694336 time for calcul the mask position with numpy : 0.0414278507232666 nb_pixel_total : 13532 time to create 1 rle with old method : 0.02025604248046875 time for calcul the mask position with numpy : 0.04379701614379883 nb_pixel_total : 156456 time to create 1 rle with new method : 1.0497407913208008 time for calcul the mask position with numpy : 0.037114858627319336 nb_pixel_total : 54643 time to create 1 rle with old method : 0.06253790855407715 time for calcul the mask position with numpy : 0.036118507385253906 nb_pixel_total : 16780 time to create 1 rle with old method : 0.026740074157714844 time for calcul the mask position with numpy : 0.042961835861206055 nb_pixel_total : 5941 time to create 1 rle with old method : 0.007934808731079102 create new chi : 8.944475650787354 time to delete rle : 0.0027031898498535156 batch 1 Loaded 24 chid ids of type : 3726 Number RLEs to save : 21778 TO DO : save crop sub photo not yet done ! save time : 6.520230293273926 nb_obj : 27 nb_hashtags : 7 time to prepare the origin masks : 11.588553667068481 time for calcul the mask position with numpy : 0.49312901496887207 nb_pixel_total : 5810883 time to create 1 rle with new method : 0.5628695487976074 time for calcul the mask position with numpy : 0.030054807662963867 nb_pixel_total : 2447 time to create 1 rle with old method : 0.0029447078704833984 time for calcul the mask position with numpy : 0.03191113471984863 nb_pixel_total : 9368 time to create 1 rle with old method : 0.011469125747680664 time for calcul the mask position with numpy : 0.0311429500579834 nb_pixel_total : 82717 time to create 1 rle with old method : 0.09980392456054688 time for calcul the mask position with numpy : 0.035681724548339844 nb_pixel_total : 68922 time to create 1 rle with old method : 0.08185124397277832 time for calcul the mask position with numpy : 0.03256058692932129 nb_pixel_total : 215234 time to create 1 rle with new method : 0.342726469039917 time for calcul the mask position with numpy : 0.03053450584411621 nb_pixel_total : 158229 time to create 1 rle with new method : 0.69374680519104 time for calcul the mask position with numpy : 0.029618024826049805 nb_pixel_total : 117671 time to create 1 rle with old method : 0.12489771842956543 time for calcul the mask position with numpy : 0.027169227600097656 nb_pixel_total : 10832 time to create 1 rle with old method : 0.011660337448120117 time for calcul the mask position with numpy : 0.027938365936279297 nb_pixel_total : 14495 time to create 1 rle with old method : 0.015263557434082031 time for calcul the mask position with numpy : 0.02705097198486328 nb_pixel_total : 5276 time to create 1 rle with old method : 0.005617618560791016 time for calcul the mask position with numpy : 0.0274808406829834 nb_pixel_total : 33855 time to create 1 rle with old method : 0.037033796310424805 time for calcul the mask position with numpy : 0.029949426651000977 nb_pixel_total : 21177 time to create 1 rle with old method : 0.0243074893951416 time for calcul the mask position with numpy : 0.02953934669494629 nb_pixel_total : 14550 time to create 1 rle with old method : 0.0162203311920166 time for calcul the mask position with numpy : 0.029798507690429688 nb_pixel_total : 110974 time to create 1 rle with old method : 0.1231544017791748 time for calcul the mask position with numpy : 0.028379440307617188 nb_pixel_total : 2146 time to create 1 rle with old method : 0.0026378631591796875 time for calcul the mask position with numpy : 0.02946758270263672 nb_pixel_total : 159102 time to create 1 rle with new method : 0.4237031936645508 time for calcul the mask position with numpy : 0.029804229736328125 nb_pixel_total : 33410 time to create 1 rle with old method : 0.0368199348449707 time for calcul the mask position with numpy : 0.030613422393798828 nb_pixel_total : 4221 time to create 1 rle with old method : 0.005080699920654297 time for calcul the mask position with numpy : 0.032782554626464844 nb_pixel_total : 5612 time to create 1 rle with old method : 0.00859212875366211 time for calcul the mask position with numpy : 0.03620719909667969 nb_pixel_total : 10443 time to create 1 rle with old method : 0.011578798294067383 time for calcul the mask position with numpy : 0.02773880958557129 nb_pixel_total : 4547 time to create 1 rle with old method : 0.004811763763427734 time for calcul the mask position with numpy : 0.02865433692932129 nb_pixel_total : 97150 time to create 1 rle with old method : 0.11106038093566895 time for calcul the mask position with numpy : 0.040825605392456055 nb_pixel_total : 24655 time to create 1 rle with old method : 0.027420759201049805 time for calcul the mask position with numpy : 0.028865337371826172 nb_pixel_total : 11398 time to create 1 rle with old method : 0.012400627136230469 time for calcul the mask position with numpy : 0.02805185317993164 nb_pixel_total : 8105 time to create 1 rle with old method : 0.009143590927124023 time for calcul the mask position with numpy : 0.02790379524230957 nb_pixel_total : 3827 time to create 1 rle with old method : 0.004077911376953125 time for calcul the mask position with numpy : 0.02820134162902832 nb_pixel_total : 8994 time to create 1 rle with old method : 0.011539220809936523 create new chi : 4.2710120677948 time to delete rle : 0.0018188953399658203 batch 1 Loaded 28 chid ids of type : 3726 Number RLEs to save : 15209 TO DO : save crop sub photo not yet done ! save time : 0.9009194374084473 map_output_result : {1339768163: (0.0, 'Should be the crop_list due to order', 0), 1339768140: (0.0, 'Should be the crop_list due to order', 0), 1339768138: (0.0, 'Should be the crop_list due to order', 0), 1339768132: (0.0, 'Should be the crop_list due to order', 0), 1339768127: (0.0, 'Should be the crop_list due to order', 0), 1339767994: (0.0, 'Should be the crop_list due to order', 0), 1339767992: (0.0, 'Should be the crop_list due to order', 0), 1339767989: (0.0, 'Should be the crop_list due to order', 0), 1339767985: (0.0, 'Should be the crop_list due to order', 0), 1339767964: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1339768163, 1339768140, 1339768138, 1339768132, 1339768127, 1339767994, 1339767992, 1339767989, 1339767985, 1339767964] Looping around the photos to save general results len do output : 10 /1339768163.Didn't retrieve data . /1339768140.Didn't retrieve data . /1339768138.Didn't retrieve data . /1339768132.Didn't retrieve data . /1339768127.Didn't retrieve data . /1339767994.Didn't retrieve data . /1339767992.Didn't retrieve data . /1339767989.Didn't retrieve data . /1339767985.Didn't retrieve data . /1339767964.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768163', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768140', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768138', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768132', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768127', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767994', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767992', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767989', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767985', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767964', None, None, None, None, None, '2619767') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.013883829116821289 save_final save missing photos in datou_result : time spend for datou_step_exec : 176.0341761112213 time spend to save output : 0.014798164367675781 total time spend for step 5 : 176.048974275589 step6:crop_condition Thu Feb 27 16:07:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 188 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 16 About to insert : list_path_to_insert length 16 new photo from crops ! About to upload 16 photos upload in portfolio : 4869462 init cache_photo without model_param we have 16 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740668869_76450 we have uploaded 16 photos in the portfolio 4869462 time of upload the photos Elapsed time : 5.7821269035339355 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! About to upload 6 photos upload in portfolio : 4869462 init cache_photo without model_param we have 6 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740668879_76450 we have uploaded 6 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.079317808151245 we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! About to upload 7 photos upload in portfolio : 4869462 init cache_photo without model_param we have 7 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740668883_76450 we have uploaded 7 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.3457419872283936 we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! About to upload 6 photos upload in portfolio : 4869462 init cache_photo without model_param we have 6 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740668888_76450 we have uploaded 6 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.7696034908294678 we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! About to upload 6 photos upload in portfolio : 4869462 init cache_photo without model_param we have 6 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740668893_76450 we have uploaded 6 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.02632999420166 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 4869462 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740668898_76450 we have uploaded 4 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.550184726715088 we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1339768163, 1339768140, 1339768138, 1339768132, 1339768127, 1339767994, 1339767992, 1339767989, 1339767985, 1339767964] Looping around the photos to save general results len do output : 45 /1339938366Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938367Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938368Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938369Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938370Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938371Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938372Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938373Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938374Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938375Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938376Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938377Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938378Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938379Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938380Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938381Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938384Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938385Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938386Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938387Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938388Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938389Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938390Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938391Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938392Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938393Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938394Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938395Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938396Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938398Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938399Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938400Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938401Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938402Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938403Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938406Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938407Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938408Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938409Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938410Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938411Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938416Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938417Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938418Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339938419Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768163', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768140', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768138', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768132', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768127', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767994', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767992', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767989', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767985', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767964', None, None, None, None, None, '2619767') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 145 time used for this insertion : 0.03126335144042969 save_final save missing photos in datou_result : time spend for datou_step_exec : 37.377657413482666 time spend to save output : 0.03257036209106445 total time spend for step 6 : 37.41022777557373 step7:ventilate_hashtags_in_portfolio Thu Feb 27 16:08:20 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 20881947 get user id for portfolio 20881947 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881947 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('kraft','flou','Carton_brun','plastique','mal_croppe','papier','Teint_Dans_La_Masse','cartonnette','autre_refus','metal','environnement','Carton_gris')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881947 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('kraft','flou','Carton_brun','plastique','mal_croppe','papier','Teint_Dans_La_Masse','cartonnette','autre_refus','metal','environnement','Carton_gris')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/20884094,20884095,20884096,20884097,20884098,20884099,20884100,20884101,20884102,20884103,20884104,20884105?tags=mal_croppe,Carton_brun,papier,environnement,kraft,metal,Teint_Dans_La_Masse,flou,autre_refus,plastique,cartonnette,Carton_gris Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1339768163, 1339768140, 1339768138, 1339768132, 1339768127, 1339767994, 1339767992, 1339767989, 1339767985, 1339767964] Looping around the photos to save general results len do output : 1 /20881947. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768163', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768140', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768138', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768132', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768127', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767994', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767992', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767989', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767985', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767964', None, None, None, None, None, '2619767') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.01314544677734375 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.4824879169464111 time spend to save output : 0.013634204864501953 total time spend for step 7 : 1.496122121810913 step8:final Thu Feb 27 16:08:21 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1339768163: ('0.08449184480973848',), 1339768140: ('0.08449184480973848',), 1339768138: ('0.08449184480973848',), 1339768132: ('0.08449184480973848',), 1339768127: ('0.08449184480973848',), 1339767994: ('0.08449184480973848',), 1339767992: ('0.08449184480973848',), 1339767989: ('0.08449184480973848',), 1339767985: ('0.08449184480973848',), 1339767964: ('0.08449184480973848',)} new output for save of step final : {1339768163: ('0.08449184480973848',), 1339768140: ('0.08449184480973848',), 1339768138: ('0.08449184480973848',), 1339768132: ('0.08449184480973848',), 1339768127: ('0.08449184480973848',), 1339767994: ('0.08449184480973848',), 1339767992: ('0.08449184480973848',), 1339767989: ('0.08449184480973848',), 1339767985: ('0.08449184480973848',), 1339767964: ('0.08449184480973848',)} [1339768163, 1339768140, 1339768138, 1339768132, 1339768127, 1339767994, 1339767992, 1339767989, 1339767985, 1339767964] Looping around the photos to save general results len do output : 10 /1339768163.Didn't retrieve data . /1339768140.Didn't retrieve data . /1339768138.Didn't retrieve data . /1339768132.Didn't retrieve data . /1339768127.Didn't retrieve data . /1339767994.Didn't retrieve data . /1339767992.Didn't retrieve data . /1339767989.Didn't retrieve data . /1339767985.Didn't retrieve data . /1339767964.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768163', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768140', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768138', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768132', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768127', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767994', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767992', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767989', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767985', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767964', None, None, None, None, None, '2619767') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.012640953063964844 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.10669469833374023 time spend to save output : 0.013525962829589844 total time spend for step 8 : 0.12022066116333008 step9:velours_tree Thu Feb 27 16:08:21 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.05728316307067871 time spend to save output : 3.337860107421875e-05 total time spend for step 9 : 0.05731654167175293 step10:send_mail_cod Thu Feb 27 16:08:21 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P20881947_27-02-2025_16_08_21.pdf 20884094 imagette208840941740668901 20884098 imagette208840981740668901 20884099 imagette208840991740668901 20884101 imagette208841011740668901 20884102 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette208841021740668901 20884103 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette208841031740668902 20884104 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette208841041740668903 20884095 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette208840951740668903 20884100 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette208841001740668904 20884105 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette208841051740668905 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20881947 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20884094,20884095,20884096,20884097,20884098,20884099,20884100,20884101,20884102,20884103,20884104,20884105?tags=mal_croppe,Carton_brun,papier,environnement,kraft,metal,Teint_Dans_La_Masse,flou,autre_refus,plastique,cartonnette,Carton_gris your option no_mail is active, we will not send the real mail to your client args[1339768163] : ('0.08449184480973848',) no score found for photo 1339768163 We are sending mail with results at report@fotonower.com args[1339768140] : ('0.08449184480973848',) no score found for photo 1339768140 We are sending mail with results at report@fotonower.com args[1339768138] : ('0.08449184480973848',) no score found for photo 1339768138 We are sending mail with results at report@fotonower.com args[1339768132] : ('0.08449184480973848',) no score found for photo 1339768132 We are sending mail with results at report@fotonower.com args[1339768127] : ('0.08449184480973848',) no score found for photo 1339768127 We are sending mail with results at report@fotonower.com args[1339767994] : ('0.08449184480973848',) no score found for photo 1339767994 We are sending mail with results at report@fotonower.com args[1339767992] : ('0.08449184480973848',) no score found for photo 1339767992 We are sending mail with results at report@fotonower.com args[1339767989] : ('0.08449184480973848',) no score found for photo 1339767989 We are sending mail with results at report@fotonower.com args[1339767985] : ('0.08449184480973848',) no score found for photo 1339767985 We are sending mail with results at report@fotonower.com args[1339767964] : ('0.08449184480973848',) no score found for photo 1339767964 We are sending mail with results at report@fotonower.com refus_total : 0.08449184480973848 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20881947 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1339803007,1339767578,1339767582,1339803004,1339802842,1339802837,1339802834,1339768174,1339767994,1339768127,1339768171,1339767571,1339768140,1339768163,1339768138,1339802998,1339767590,1339768168,1339768132,1339767992) Found this number of photos: 20 begin to download photo : 1339803007 begin to download photo : 1339802837 begin to download photo : 1339768171 begin to download photo : 1339802998 download finish for photo 1339802998 begin to download photo : 1339767590 download finish for photo 1339803007 begin to download photo : 1339767578 download finish for photo 1339802837 begin to download photo : 1339802834 download finish for photo 1339768171 begin to download photo : 1339767571 download finish for photo 1339767590 begin to download photo : 1339768168 download finish for photo 1339767571 begin to download photo : 1339768140 download finish for photo 1339802834 begin to download photo : 1339768174 download finish for photo 1339767578 begin to download photo : 1339767582 download finish for photo 1339768168 begin to download photo : 1339768132 download finish for photo 1339767582 begin to download photo : 1339803004 download finish for photo 1339768174 begin to download photo : 1339767994 download finish for photo 1339768140 begin to download photo : 1339768163 download finish for photo 1339768132 begin to download photo : 1339767992 download finish for photo 1339803004 begin to download photo : 1339802842 download finish for photo 1339767994 begin to download photo : 1339768127 download finish for photo 1339767992 download finish for photo 1339768163 begin to download photo : 1339768138 download finish for photo 1339802842 download finish for photo 1339768138 download finish for photo 1339768127 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881947_27-02-2025_16_08_21.pdf results_Auto_P20881947_27-02-2025_16_08_21.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881947_27-02-2025_16_08_21.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3459','20881947','results_Auto_P20881947_27-02-2025_16_08_21.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881947_27-02-2025_16_08_21.pdf','pdf','','1.06','0.08449184480973848') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1339768163, 1339768140, 1339768138, 1339768132, 1339768127, 1339767994, 1339767992, 1339767989, 1339767985, 1339767964] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768163', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768140', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768138', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768132', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768127', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767994', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767992', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767989', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767985', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767964', None, None, None, None, None, '2619767') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.013886451721191406 save_final save missing photos in datou_result : time spend for datou_step_exec : 8.673423767089844 time spend to save output : 0.014222860336303711 total time spend for step 10 : 8.687646627426147 step11:split_time_score Thu Feb 27 16:08:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('18', 26),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 26022025 20881947 Nombre de photos uploadées : 26 / 23040 (0%) 26022025 20881947 Nombre de photos taguées (types de déchets): 0 / 26 (0%) 26022025 20881947 Nombre de photos taguées (volume) : 0 / 26 (0%) elapsed_time : load_data_split_time_score 6.198883056640625e-06 elapsed_time : order_list_meta_photo_and_scores 1.6689300537109375e-05 ?????????????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0026674270629882812 elapsed_time : insert_dashboard_record_day_entry 0.15269780158996582 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881903 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881904 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881905 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881906 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881907 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881908 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881909 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881910 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881911 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881912 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881913 order by id desc limit 1 Qualite : 0.063119307517658 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881914_27-02-2025_15_01_14.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881914 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881914 AND mptpi.`type`=3726 To do Qualite : 0.15187597077362103 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881915_27-02-2025_15_26_35.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881915 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881915 AND mptpi.`type`=3327 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881916 order by id desc limit 1 Qualite : 0.20532825832879442 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881917_27-02-2025_15_56_43.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881917 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881917 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881918 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881919 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881920 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881921 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881922 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881923 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881924 order by id desc limit 1 Qualite : 0.17112767792302108 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881925_27-02-2025_15_18_07.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881925 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881925 AND mptpi.`type`=3594 To do Qualite : 0.08562360163737219 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881926_27-02-2025_14_57_30.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881926 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881926 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881927 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881928 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881929 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881930 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881931 order by id desc limit 1 Qualite : 0.3424234391415626 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881932_27-02-2025_15_47_33.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881932 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881932 AND mptpi.`type`=4200 To do Qualite : 0.08025362176513684 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881933_27-02-2025_15_44_55.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881933 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11524 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11533 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11532 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11528 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11528 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11531 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11531 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11525 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11525 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11578 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11578 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11527 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11526 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11526 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11535 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11527 doesn't seem to be define in the database( WARNING : type of input 3 of step 11526 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11524 doesn't seem to be define in the database( WARNING : type of input 2 of step 11528 doesn't seem to be define in the database( WARNING : output 1 of step 11524 have datatype=2 whereas input 1 of step 11531 have datatype=7 WARNING : type of output 2 of step 11531 doesn't seem to be define in the database( WARNING : type of input 1 of step 11525 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11527 have datatype=10 whereas input 3 of step 11534 have datatype=6 WARNING : type of input 2 of step 11578 doesn't seem to be define in the database( WARNING : output 1 of step 11525 have datatype=7 whereas input 2 of step 11578 have datatype=None WARNING : type of output 3 of step 11578 doesn't seem to be define in the database( WARNING : type of input 1 of step 11527 doesn't seem to be define in the database( WARNING : output 0 of step 11527 have datatype=10 whereas input 0 of step 11584 have datatype=18 WARNING : type of input 5 of step 11534 doesn't seem to be define in the database( WARNING : output 0 of step 11584 have datatype=11 whereas input 5 of step 11534 have datatype=None WARNING : type of output 1 of step 11533 doesn't seem to be define in the database( WARNING : type of input 3 of step 11528 doesn't seem to be define in the database( WARNING : type of output 1 of step 11532 doesn't seem to be define in the database( WARNING : type of input 3 of step 11528 doesn't seem to be define in the database( WARNING : output 0 of step 11531 have datatype=1 whereas input 0 of step 11525 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881933 AND mptpi.`type`=4211 To do Qualite : 0.2121681261984472 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881934_27-02-2025_16_02_51.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881934 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881934 AND mptpi.`type`=3327 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881935 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881936 order by id desc limit 1 Qualite : 0.2051292416450198 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881937_27-02-2025_15_03_53.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881937 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881937 AND mptpi.`type`=3594 To do Qualite : 0.22323044144596488 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881938_27-02-2025_15_18_17.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881938 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881938 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881939 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881940 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881941 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881942 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881943 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881944 order by id desc limit 1 Qualite : 0.04719140436512212 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881945_27-02-2025_16_02_11.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881945 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881945 AND mptpi.`type`=4207 To do Qualite : 0.06795947229191704 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881946_27-02-2025_15_10_24.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881946 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881946 AND mptpi.`type`=4207 To do Qualite : 0.07376039365091426 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881947_27-02-2025_16_08_21.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881947 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881947 AND mptpi.`type`=3726 To do Qualite : 0.15940719748547566 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881948_27-02-2025_14_47_18.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881948 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881948 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881949 order by id desc limit 1 Qualite : 0.5400794373803116 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881950_27-02-2025_14_35_28.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881950 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881950 AND mptpi.`type`=4200 To do Qualite : 0.18727468064128952 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881951 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881951 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881952 order by id desc limit 1 Qualite : 0.05315619982736321 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881953_27-02-2025_14_56_05.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881953 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881953 AND mptpi.`type`=4207 To do Qualite : 0.09784591310224797 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881954_27-02-2025_15_55_37.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881954 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881954 AND mptpi.`type`=4207 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'26022025': {'nb_upload': 26, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1339768163, 1339768140, 1339768138, 1339768132, 1339768127, 1339767994, 1339767992, 1339767989, 1339767985, 1339767964] Looping around the photos to save general results len do output : 1 /20881947Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768163', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768140', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768138', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768132', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339768127', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767994', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767992', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767989', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767985', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', None, '1339767964', None, None, None, None, None, '2619767') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.013323545455932617 save_final save missing photos in datou_result : time spend for datou_step_exec : 29.557409048080444 time spend to save output : 0.01378011703491211 total time spend for step 11 : 29.571189165115356 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 179.92user 152.99system 6:35.50elapsed 84%CPU (0avgtext+0avgdata 4186048maxresident)k 2959264inputs+162752outputs (12272major+17616343minor)pagefaults 0swaps