python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 3459 -l 10 ' -s datou_current_3459 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 284429 load datou : 3459 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) load thcls load THCL from format json or kwargs add thcl : 2456 in CacheModelConfig load THCL from format json or kwargs add thcl : 2868 in CacheModelConfig load THCL from format json or kwargs add thcl : 2896 in CacheModelConfig load pdts add pdt : 4999 in CacheModelConfig add pdt : 5288 in CacheModelConfig add pdt : 5309 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 we have a portfolio with more photos than limit : 22>10 please execute split_portfolio.py -i 20341531 -l 10 size over we load limit photo not treated list_input_json: [] Current got : datou_id : 3459, datou_cur_ids : ['2565109'] with mtr_portfolio_ids : ['20341531'] and first list_photo_ids : [] new path : /proc/284429/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, thcl, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 10 ; length of list_pids : 10 ; length of list_args : 10 time to download the photos : 1.865797758102417 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Fri Feb 7 16:54:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10774 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-07 16:54:36.041687: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-07 16:54:36.075138: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-07 16:54:36.077559: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f4748000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-07 16:54:36.077620: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-07 16:54:36.083713: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-07 16:54:36.325452: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x40998200 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-07 16:54:36.325504: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-07 16:54:36.327404: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-07 16:54:36.328638: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-07 16:54:36.334682: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-07 16:54:36.357005: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-07 16:54:36.358997: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-07 16:54:36.396010: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-07 16:54:36.402304: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-07 16:54:36.466921: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-07 16:54:36.468764: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-07 16:54:36.469198: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-07 16:54:36.470484: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-07 16:54:36.470504: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-07 16:54:36.470516: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-07 16:54:36.472618: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9985 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-07 16:54:36.821315: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-07 16:54:36.821445: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-07 16:54:36.821462: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-07 16:54:36.821477: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-07 16:54:36.821491: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-07 16:54:36.821505: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-07 16:54:36.821518: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-07 16:54:36.821533: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-07 16:54:36.822714: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-07 16:54:36.823932: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-07 16:54:36.823968: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-07 16:54:36.823983: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-07 16:54:36.823997: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-07 16:54:36.824011: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-07 16:54:36.824025: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-07 16:54:36.824039: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-07 16:54:36.824058: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-07 16:54:36.825324: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-07 16:54:36.825360: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-07 16:54:36.825368: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-07 16:54:36.825376: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-07 16:54:36.826610: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9985 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-07 16:54:47.850247: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-07 16:54:48.227807: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 10 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 14 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 12 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 20 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 16 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 12 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 18 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 26 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 15 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 11 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 19 Detection mask done ! Trying to reset tf kernel 284853 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5485 tf kernel not reseted sub process len(results) : 10 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 10 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10774 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.0008249282836914062 nb_pixel_total : 33763 time to create 1 rle with old method : 0.041002750396728516 length of segment : 337 time for calcul the mask position with numpy : 0.0003027915954589844 nb_pixel_total : 9450 time to create 1 rle with old method : 0.012424468994140625 length of segment : 162 time for calcul the mask position with numpy : 0.00024437904357910156 nb_pixel_total : 15493 time to create 1 rle with old method : 0.017519474029541016 length of segment : 111 time for calcul the mask position with numpy : 0.00023436546325683594 nb_pixel_total : 12276 time to create 1 rle with old method : 0.01341390609741211 length of segment : 171 time for calcul the mask position with numpy : 0.00347900390625 nb_pixel_total : 146205 time to create 1 rle with old method : 0.15490174293518066 length of segment : 566 time for calcul the mask position with numpy : 0.002975940704345703 nb_pixel_total : 136658 time to create 1 rle with old method : 0.15355920791625977 length of segment : 489 time for calcul the mask position with numpy : 0.00010776519775390625 nb_pixel_total : 3400 time to create 1 rle with old method : 0.003788471221923828 length of segment : 93 time for calcul the mask position with numpy : 0.000270843505859375 nb_pixel_total : 15928 time to create 1 rle with old method : 0.017750263214111328 length of segment : 160 time for calcul the mask position with numpy : 0.0003044605255126953 nb_pixel_total : 19088 time to create 1 rle with old method : 0.02002263069152832 length of segment : 233 time for calcul the mask position with numpy : 0.0001392364501953125 nb_pixel_total : 6363 time to create 1 rle with old method : 0.0071141719818115234 length of segment : 144 time for calcul the mask position with numpy : 0.0019049644470214844 nb_pixel_total : 119653 time to create 1 rle with old method : 0.1692342758178711 length of segment : 551 time for calcul the mask position with numpy : 0.0010936260223388672 nb_pixel_total : 79600 time to create 1 rle with old method : 0.08843255043029785 length of segment : 348 time for calcul the mask position with numpy : 0.00045371055603027344 nb_pixel_total : 19969 time to create 1 rle with old method : 0.02613663673400879 length of segment : 203 time for calcul the mask position with numpy : 0.0007648468017578125 nb_pixel_total : 27935 time to create 1 rle with old method : 0.030483245849609375 length of segment : 477 time for calcul the mask position with numpy : 0.0028524398803710938 nb_pixel_total : 196481 time to create 1 rle with new method : 0.011835336685180664 length of segment : 542 time for calcul the mask position with numpy : 0.00019598007202148438 nb_pixel_total : 2737 time to create 1 rle with old method : 0.007394075393676758 length of segment : 53 time for calcul the mask position with numpy : 0.00018358230590820312 nb_pixel_total : 7641 time to create 1 rle with old method : 0.009404659271240234 length of segment : 217 time for calcul the mask position with numpy : 0.0008883476257324219 nb_pixel_total : 48676 time to create 1 rle with old method : 0.05945563316345215 length of segment : 318 time for calcul the mask position with numpy : 0.00022268295288085938 nb_pixel_total : 9144 time to create 1 rle with old method : 0.012756109237670898 length of segment : 134 time for calcul the mask position with numpy : 0.0058748722076416016 nb_pixel_total : 281535 time to create 1 rle with new method : 0.013644695281982422 length of segment : 698 time for calcul the mask position with numpy : 0.0003352165222167969 nb_pixel_total : 7096 time to create 1 rle with old method : 0.007787227630615234 length of segment : 111 time for calcul the mask position with numpy : 0.00823664665222168 nb_pixel_total : 289886 time to create 1 rle with new method : 0.011961698532104492 length of segment : 733 time for calcul the mask position with numpy : 0.00017952919006347656 nb_pixel_total : 2407 time to create 1 rle with old method : 0.0026865005493164062 length of segment : 73 time for calcul the mask position with numpy : 0.0003693103790283203 nb_pixel_total : 10957 time to create 1 rle with old method : 0.012163639068603516 length of segment : 97 time for calcul the mask position with numpy : 0.00021076202392578125 nb_pixel_total : 3957 time to create 1 rle with old method : 0.0047185420989990234 length of segment : 68 time for calcul the mask position with numpy : 0.0011560916900634766 nb_pixel_total : 35491 time to create 1 rle with old method : 0.038745880126953125 length of segment : 235 time for calcul the mask position with numpy : 0.0015137195587158203 nb_pixel_total : 41892 time to create 1 rle with old method : 0.04468059539794922 length of segment : 215 time for calcul the mask position with numpy : 0.000598907470703125 nb_pixel_total : 16202 time to create 1 rle with old method : 0.017187118530273438 length of segment : 214 time for calcul the mask position with numpy : 0.00020837783813476562 nb_pixel_total : 2997 time to create 1 rle with old method : 0.0032956600189208984 length of segment : 74 time for calcul the mask position with numpy : 0.00015020370483398438 nb_pixel_total : 2065 time to create 1 rle with old method : 0.0024917125701904297 length of segment : 46 time for calcul the mask position with numpy : 0.0019376277923583984 nb_pixel_total : 56473 time to create 1 rle with old method : 0.05853128433227539 length of segment : 705 time for calcul the mask position with numpy : 0.00016617774963378906 nb_pixel_total : 2070 time to create 1 rle with old method : 0.0024437904357910156 length of segment : 68 time for calcul the mask position with numpy : 0.0021266937255859375 nb_pixel_total : 90099 time to create 1 rle with old method : 0.09630894660949707 length of segment : 259 time for calcul the mask position with numpy : 0.0035293102264404297 nb_pixel_total : 162305 time to create 1 rle with new method : 0.008657217025756836 length of segment : 344 time for calcul the mask position with numpy : 0.0011239051818847656 nb_pixel_total : 28677 time to create 1 rle with old method : 0.03041863441467285 length of segment : 305 time for calcul the mask position with numpy : 0.0014777183532714844 nb_pixel_total : 32193 time to create 1 rle with old method : 0.03335881233215332 length of segment : 376 time for calcul the mask position with numpy : 0.0007543563842773438 nb_pixel_total : 12485 time to create 1 rle with old method : 0.013358354568481445 length of segment : 486 time for calcul the mask position with numpy : 0.0002620220184326172 nb_pixel_total : 2365 time to create 1 rle with old method : 0.0027527809143066406 length of segment : 84 time for calcul the mask position with numpy : 0.0011250972747802734 nb_pixel_total : 87770 time to create 1 rle with old method : 0.09541463851928711 length of segment : 258 time for calcul the mask position with numpy : 9.894371032714844e-05 nb_pixel_total : 2140 time to create 1 rle with old method : 0.0026199817657470703 length of segment : 75 time for calcul the mask position with numpy : 0.003568887710571289 nb_pixel_total : 105022 time to create 1 rle with old method : 0.11415672302246094 length of segment : 465 time for calcul the mask position with numpy : 0.0010237693786621094 nb_pixel_total : 34720 time to create 1 rle with old method : 0.03648066520690918 length of segment : 492 time for calcul the mask position with numpy : 0.00024247169494628906 nb_pixel_total : 3373 time to create 1 rle with old method : 0.0038030147552490234 length of segment : 107 time for calcul the mask position with numpy : 0.002607107162475586 nb_pixel_total : 118299 time to create 1 rle with old method : 0.12660646438598633 length of segment : 387 time for calcul the mask position with numpy : 0.0008144378662109375 nb_pixel_total : 25094 time to create 1 rle with old method : 0.02657794952392578 length of segment : 198 time for calcul the mask position with numpy : 0.00012993812561035156 nb_pixel_total : 3821 time to create 1 rle with old method : 0.004441976547241211 length of segment : 71 time for calcul the mask position with numpy : 0.0022306442260742188 nb_pixel_total : 93121 time to create 1 rle with old method : 0.09924149513244629 length of segment : 316 time for calcul the mask position with numpy : 0.0003077983856201172 nb_pixel_total : 5230 time to create 1 rle with old method : 0.0061910152435302734 length of segment : 140 time for calcul the mask position with numpy : 0.007909297943115234 nb_pixel_total : 305726 time to create 1 rle with new method : 0.015592575073242188 length of segment : 765 time for calcul the mask position with numpy : 0.0014150142669677734 nb_pixel_total : 58285 time to create 1 rle with old method : 0.06310224533081055 length of segment : 247 time for calcul the mask position with numpy : 0.0002696514129638672 nb_pixel_total : 2345 time to create 1 rle with old method : 0.002741098403930664 length of segment : 109 time for calcul the mask position with numpy : 0.0002789497375488281 nb_pixel_total : 10336 time to create 1 rle with old method : 0.011642932891845703 length of segment : 221 time for calcul the mask position with numpy : 0.000957489013671875 nb_pixel_total : 28920 time to create 1 rle with old method : 0.03166651725769043 length of segment : 243 time for calcul the mask position with numpy : 0.0008788108825683594 nb_pixel_total : 55842 time to create 1 rle with old method : 0.06301069259643555 length of segment : 270 time for calcul the mask position with numpy : 0.00024127960205078125 nb_pixel_total : 3721 time to create 1 rle with old method : 0.0043909549713134766 length of segment : 118 time for calcul the mask position with numpy : 0.002207040786743164 nb_pixel_total : 36134 time to create 1 rle with old method : 0.043396949768066406 length of segment : 252 time for calcul the mask position with numpy : 0.0014028549194335938 nb_pixel_total : 60218 time to create 1 rle with old method : 0.06592845916748047 length of segment : 272 time for calcul the mask position with numpy : 0.00034499168395996094 nb_pixel_total : 6020 time to create 1 rle with old method : 0.006946563720703125 length of segment : 131 time for calcul the mask position with numpy : 0.0013895034790039062 nb_pixel_total : 59726 time to create 1 rle with old method : 0.06625604629516602 length of segment : 319 time for calcul the mask position with numpy : 0.003529071807861328 nb_pixel_total : 151738 time to create 1 rle with new method : 0.0066487789154052734 length of segment : 515 time for calcul the mask position with numpy : 0.0022797584533691406 nb_pixel_total : 57540 time to create 1 rle with old method : 0.06192898750305176 length of segment : 600 time for calcul the mask position with numpy : 0.000385284423828125 nb_pixel_total : 8664 time to create 1 rle with old method : 0.010113239288330078 length of segment : 202 time for calcul the mask position with numpy : 0.0008118152618408203 nb_pixel_total : 29779 time to create 1 rle with old method : 0.03349709510803223 length of segment : 187 time for calcul the mask position with numpy : 0.005182743072509766 nb_pixel_total : 6776 time to create 1 rle with old method : 0.009949207305908203 length of segment : 52 time for calcul the mask position with numpy : 0.0016775131225585938 nb_pixel_total : 56448 time to create 1 rle with old method : 0.06207776069641113 length of segment : 265 time for calcul the mask position with numpy : 0.001691579818725586 nb_pixel_total : 33767 time to create 1 rle with old method : 0.037441253662109375 length of segment : 465 time for calcul the mask position with numpy : 0.0008490085601806641 nb_pixel_total : 26065 time to create 1 rle with old method : 0.029854297637939453 length of segment : 363 time for calcul the mask position with numpy : 0.0003991127014160156 nb_pixel_total : 19412 time to create 1 rle with old method : 0.022114276885986328 length of segment : 144 time for calcul the mask position with numpy : 0.0003859996795654297 nb_pixel_total : 12714 time to create 1 rle with old method : 0.014375925064086914 length of segment : 98 time for calcul the mask position with numpy : 0.0007536411285400391 nb_pixel_total : 24735 time to create 1 rle with old method : 0.027048349380493164 length of segment : 166 time for calcul the mask position with numpy : 0.001432180404663086 nb_pixel_total : 59595 time to create 1 rle with old method : 0.062111616134643555 length of segment : 508 time for calcul the mask position with numpy : 0.0002925395965576172 nb_pixel_total : 4640 time to create 1 rle with old method : 0.005265474319458008 length of segment : 115 time for calcul the mask position with numpy : 0.001131296157836914 nb_pixel_total : 38601 time to create 1 rle with old method : 0.042951345443725586 length of segment : 237 time for calcul the mask position with numpy : 0.00043129920959472656 nb_pixel_total : 6993 time to create 1 rle with old method : 0.011394500732421875 length of segment : 134 time for calcul the mask position with numpy : 0.0029265880584716797 nb_pixel_total : 66824 time to create 1 rle with old method : 0.0886683464050293 length of segment : 665 time for calcul the mask position with numpy : 0.0016527175903320312 nb_pixel_total : 53400 time to create 1 rle with old method : 0.05899858474731445 length of segment : 318 time for calcul the mask position with numpy : 0.0009503364562988281 nb_pixel_total : 21388 time to create 1 rle with old method : 0.023099422454833984 length of segment : 346 time for calcul the mask position with numpy : 0.003979682922363281 nb_pixel_total : 126988 time to create 1 rle with old method : 0.13738441467285156 length of segment : 536 time for calcul the mask position with numpy : 0.0028781890869140625 nb_pixel_total : 165515 time to create 1 rle with new method : 0.007844686508178711 length of segment : 666 time for calcul the mask position with numpy : 0.002382993698120117 nb_pixel_total : 106533 time to create 1 rle with old method : 0.11251449584960938 length of segment : 429 time for calcul the mask position with numpy : 0.0021953582763671875 nb_pixel_total : 79711 time to create 1 rle with old method : 0.08837699890136719 length of segment : 403 time for calcul the mask position with numpy : 0.0001239776611328125 nb_pixel_total : 1684 time to create 1 rle with old method : 0.0020847320556640625 length of segment : 52 time for calcul the mask position with numpy : 0.003387928009033203 nb_pixel_total : 146818 time to create 1 rle with old method : 0.15936779975891113 length of segment : 577 time for calcul the mask position with numpy : 0.003317117691040039 nb_pixel_total : 52949 time to create 1 rle with old method : 0.05809640884399414 length of segment : 451 time for calcul the mask position with numpy : 0.0006582736968994141 nb_pixel_total : 21193 time to create 1 rle with old method : 0.023549556732177734 length of segment : 169 time for calcul the mask position with numpy : 0.001798391342163086 nb_pixel_total : 54901 time to create 1 rle with old method : 0.061239004135131836 length of segment : 395 time for calcul the mask position with numpy : 0.004870414733886719 nb_pixel_total : 67061 time to create 1 rle with old method : 0.08532357215881348 length of segment : 464 time for calcul the mask position with numpy : 0.0009770393371582031 nb_pixel_total : 35923 time to create 1 rle with old method : 0.03989744186401367 length of segment : 133 time for calcul the mask position with numpy : 0.0016863346099853516 nb_pixel_total : 38094 time to create 1 rle with old method : 0.041880130767822266 length of segment : 309 time for calcul the mask position with numpy : 0.0003693103790283203 nb_pixel_total : 9876 time to create 1 rle with old method : 0.011394739151000977 length of segment : 142 time for calcul the mask position with numpy : 0.0010042190551757812 nb_pixel_total : 22409 time to create 1 rle with old method : 0.025096416473388672 length of segment : 226 time for calcul the mask position with numpy : 0.0005624294281005859 nb_pixel_total : 13139 time to create 1 rle with old method : 0.015169858932495117 length of segment : 130 time for calcul the mask position with numpy : 0.0001609325408935547 nb_pixel_total : 1756 time to create 1 rle with old method : 0.0021905899047851562 length of segment : 50 time for calcul the mask position with numpy : 0.0013518333435058594 nb_pixel_total : 32824 time to create 1 rle with old method : 0.037331342697143555 length of segment : 183 time for calcul the mask position with numpy : 0.0002808570861816406 nb_pixel_total : 5063 time to create 1 rle with old method : 0.005989789962768555 length of segment : 92 time for calcul the mask position with numpy : 0.0002009868621826172 nb_pixel_total : 2527 time to create 1 rle with old method : 0.003114938735961914 length of segment : 56 time for calcul the mask position with numpy : 0.002359151840209961 nb_pixel_total : 54916 time to create 1 rle with old method : 0.06153225898742676 length of segment : 303 time for calcul the mask position with numpy : 0.0007822513580322266 nb_pixel_total : 51371 time to create 1 rle with old method : 0.05598139762878418 length of segment : 294 time for calcul the mask position with numpy : 0.00040984153747558594 nb_pixel_total : 6726 time to create 1 rle with old method : 0.0077991485595703125 length of segment : 96 time for calcul the mask position with numpy : 0.009909868240356445 nb_pixel_total : 367892 time to create 1 rle with new method : 0.014803171157836914 length of segment : 509 time for calcul the mask position with numpy : 0.0015010833740234375 nb_pixel_total : 34715 time to create 1 rle with old method : 0.037795305252075195 length of segment : 292 time for calcul the mask position with numpy : 0.0006277561187744141 nb_pixel_total : 12037 time to create 1 rle with old method : 0.019555330276489258 length of segment : 216 time for calcul the mask position with numpy : 0.001226186752319336 nb_pixel_total : 27604 time to create 1 rle with old method : 0.03145170211791992 length of segment : 221 time for calcul the mask position with numpy : 0.0005331039428710938 nb_pixel_total : 23266 time to create 1 rle with old method : 0.02758193016052246 length of segment : 230 time for calcul the mask position with numpy : 0.0007851123809814453 nb_pixel_total : 25504 time to create 1 rle with old method : 0.02915334701538086 length of segment : 159 time for calcul the mask position with numpy : 0.0012547969818115234 nb_pixel_total : 29516 time to create 1 rle with old method : 0.03318381309509277 length of segment : 214 time for calcul the mask position with numpy : 0.0018880367279052734 nb_pixel_total : 50589 time to create 1 rle with old method : 0.07861995697021484 length of segment : 458 time for calcul the mask position with numpy : 0.002456188201904297 nb_pixel_total : 92236 time to create 1 rle with old method : 0.11199712753295898 length of segment : 283 time for calcul the mask position with numpy : 0.0008652210235595703 nb_pixel_total : 20390 time to create 1 rle with old method : 0.02266240119934082 length of segment : 214 time for calcul the mask position with numpy : 0.0003082752227783203 nb_pixel_total : 5963 time to create 1 rle with old method : 0.006934404373168945 length of segment : 105 time for calcul the mask position with numpy : 0.001965761184692383 nb_pixel_total : 51188 time to create 1 rle with old method : 0.05737805366516113 length of segment : 334 time for calcul the mask position with numpy : 0.0006012916564941406 nb_pixel_total : 14769 time to create 1 rle with old method : 0.02420639991760254 length of segment : 131 time for calcul the mask position with numpy : 0.0008392333984375 nb_pixel_total : 24686 time to create 1 rle with old method : 0.027436017990112305 length of segment : 173 time for calcul the mask position with numpy : 0.0020058155059814453 nb_pixel_total : 46553 time to create 1 rle with old method : 0.052826642990112305 length of segment : 348 time for calcul the mask position with numpy : 0.0017817020416259766 nb_pixel_total : 58432 time to create 1 rle with old method : 0.0684823989868164 length of segment : 668 time for calcul the mask position with numpy : 0.0008285045623779297 nb_pixel_total : 17558 time to create 1 rle with old method : 0.01980423927307129 length of segment : 193 time for calcul the mask position with numpy : 0.0002052783966064453 nb_pixel_total : 1181 time to create 1 rle with old method : 0.0014848709106445312 length of segment : 50 time for calcul the mask position with numpy : 0.0011906623840332031 nb_pixel_total : 26263 time to create 1 rle with old method : 0.029902935028076172 length of segment : 173 time for calcul the mask position with numpy : 0.00036716461181640625 nb_pixel_total : 6840 time to create 1 rle with old method : 0.008821487426757812 length of segment : 107 time for calcul the mask position with numpy : 0.0003314018249511719 nb_pixel_total : 9386 time to create 1 rle with old method : 0.010633468627929688 length of segment : 101 time for calcul the mask position with numpy : 0.0005581378936767578 nb_pixel_total : 12403 time to create 1 rle with old method : 0.013859987258911133 length of segment : 179 time for calcul the mask position with numpy : 0.0003082752227783203 nb_pixel_total : 5813 time to create 1 rle with old method : 0.006779193878173828 length of segment : 128 time for calcul the mask position with numpy : 0.001016378402709961 nb_pixel_total : 25400 time to create 1 rle with old method : 0.02768397331237793 length of segment : 391 time for calcul the mask position with numpy : 0.0028989315032958984 nb_pixel_total : 127274 time to create 1 rle with old method : 0.1355597972869873 length of segment : 565 time for calcul the mask position with numpy : 0.0011780261993408203 nb_pixel_total : 24361 time to create 1 rle with old method : 0.0275115966796875 length of segment : 265 time for calcul the mask position with numpy : 0.00019478797912597656 nb_pixel_total : 2180 time to create 1 rle with old method : 0.0024564266204833984 length of segment : 87 time for calcul the mask position with numpy : 0.0003421306610107422 nb_pixel_total : 9040 time to create 1 rle with old method : 0.009948015213012695 length of segment : 127 time for calcul the mask position with numpy : 0.0003325939178466797 nb_pixel_total : 9091 time to create 1 rle with old method : 0.010129690170288086 length of segment : 126 time for calcul the mask position with numpy : 0.0006506443023681641 nb_pixel_total : 37040 time to create 1 rle with old method : 0.03876805305480957 length of segment : 304 time for calcul the mask position with numpy : 0.0005357265472412109 nb_pixel_total : 25913 time to create 1 rle with old method : 0.028192520141601562 length of segment : 409 time for calcul the mask position with numpy : 0.002466917037963867 nb_pixel_total : 77714 time to create 1 rle with old method : 0.08469009399414062 length of segment : 560 time for calcul the mask position with numpy : 0.00031113624572753906 nb_pixel_total : 5389 time to create 1 rle with old method : 0.006343364715576172 length of segment : 87 time for calcul the mask position with numpy : 0.0003285408020019531 nb_pixel_total : 4854 time to create 1 rle with old method : 0.007983922958374023 length of segment : 90 time for calcul the mask position with numpy : 0.000985860824584961 nb_pixel_total : 25043 time to create 1 rle with old method : 0.0400538444519043 length of segment : 220 time for calcul the mask position with numpy : 0.0011444091796875 nb_pixel_total : 35390 time to create 1 rle with old method : 0.03929877281188965 length of segment : 262 time for calcul the mask position with numpy : 0.0006852149963378906 nb_pixel_total : 18786 time to create 1 rle with old method : 0.0213620662689209 length of segment : 197 time for calcul the mask position with numpy : 0.0009486675262451172 nb_pixel_total : 17432 time to create 1 rle with old method : 0.02042841911315918 length of segment : 206 time for calcul the mask position with numpy : 0.010922908782958984 nb_pixel_total : 316467 time to create 1 rle with new method : 1.0932796001434326 length of segment : 1339 time for calcul the mask position with numpy : 0.0016531944274902344 nb_pixel_total : 51258 time to create 1 rle with old method : 0.05656599998474121 length of segment : 257 time for calcul the mask position with numpy : 0.0022840499877929688 nb_pixel_total : 58235 time to create 1 rle with old method : 0.06466889381408691 length of segment : 536 time for calcul the mask position with numpy : 0.002394437789916992 nb_pixel_total : 51479 time to create 1 rle with old method : 0.05957174301147461 length of segment : 337 time for calcul the mask position with numpy : 0.0005860328674316406 nb_pixel_total : 19101 time to create 1 rle with old method : 0.021945953369140625 length of segment : 155 time for calcul the mask position with numpy : 0.0016994476318359375 nb_pixel_total : 62933 time to create 1 rle with old method : 0.07717680931091309 length of segment : 364 time for calcul the mask position with numpy : 0.0011165142059326172 nb_pixel_total : 22998 time to create 1 rle with old method : 0.026662349700927734 length of segment : 166 time for calcul the mask position with numpy : 0.0008087158203125 nb_pixel_total : 26583 time to create 1 rle with old method : 0.03269505500793457 length of segment : 227 time for calcul the mask position with numpy : 0.009201765060424805 nb_pixel_total : 242547 time to create 1 rle with new method : 0.024019718170166016 length of segment : 819 time for calcul the mask position with numpy : 0.005662202835083008 nb_pixel_total : 189252 time to create 1 rle with new method : 0.008439302444458008 length of segment : 366 time spent for convertir_results : 12.800627708435059 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 164 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 41440 save missing photos in datou_result : time spend for datou_step_exec : 58.59442400932312 time spend to save output : 6.90086817741394 total time spend for step 1 : 65.49529218673706 step2:crop_condition Fri Feb 7 16:55:37 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 164 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 10 About to insert : list_path_to_insert length 10 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 10 About to insert : list_path_to_insert length 10 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 9 About to insert : list_path_to_insert length 9 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 49 About to insert : list_path_to_insert length 49 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 9 About to insert : list_path_to_insert length 9 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1334879932, 1334879871, 1334879795, 1334879706, 1334879642, 1334879629, 1334879557, 1334879533, 1334879425, 1334879352] Looping around the photos to save general results len do output : 93 /-3656791899Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800744Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800769Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800775Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800777Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800794Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800789Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800806Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800845Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800834Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656791841Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656791862Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656791865Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3657189462Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800747Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800782Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800799Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800813Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800830Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800841Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656791869Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656791900Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800721Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800786Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800803Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800800Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800796Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800809Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3657192334Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656791844Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656791842Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656791867Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656791859Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656791861Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656791904Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660098419Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656791906Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656791893Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656791902Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656791901Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656791894Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800724Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800728Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800725Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800720Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800718Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800727Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800748Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800749Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800743Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3657192330Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800774Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800783Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800781Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800776Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800778Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800770Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800784Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800787Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800801Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800791Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800785Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800807Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800815Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656917417Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800825Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800824Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800829Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800823Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800840Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800838Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800839Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800831Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800837Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656917420Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656917421Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800842Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800846Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656791840Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656791863Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656791895Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656791896Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656791903Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800741Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800790Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800797Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800805Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656791847Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800779Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800772Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800793Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656791909Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656800808Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879932', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879871', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879795', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879706', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879642', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879629', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879557', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879533', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879425', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879352', None, None, None, None, None, '2565109') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 289 time used for this insertion : 0.33631205558776855 save_final save missing photos in datou_result : time spend for datou_step_exec : 36.01308536529541 time spend to save output : 0.3400723934173584 total time spend for step 2 : 36.35315775871277 step3:thcl Fri Feb 7 16:56:13 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 2 we are using the classfication for multi_thcl [2456, 2868] time to import caffe and check if the image exist : 0.011955738067626953 time to convert the images to numpy array : 0.043617963790893555 time to import caffe and check if the image exist : 0.01774311065673828 time to convert the images to numpy array : 0.10698509216308594 time to import caffe and check if the image exist : 0.014968395233154297 time to convert the images to numpy array : 0.1214444637298584 time to import caffe and check if the image exist : 0.015088796615600586 time to convert the images to numpy array : 0.13563132286071777 time to import caffe and check if the image exist : 0.017493724822998047 time to convert the images to numpy array : 0.14820408821105957 time to import caffe and check if the image exist : 0.021222829818725586 time to convert the images to numpy array : 0.14838290214538574 time to import caffe and check if the image exist : 0.017208099365234375 time to convert the images to numpy array : 0.15518951416015625 time to import caffe and check if the image exist : 0.018161535263061523 time to convert the images to numpy array : 0.16529250144958496 time to import caffe and check if the image exist : 0.015560388565063477 time to convert the images to numpy array : 0.17555785179138184 time to import caffe and check if the image exist : 0.014506340026855469 time to convert the images to numpy array : 0.18369770050048828 total time to convert the images to numpy array : 0.5934426784515381 list photo_ids error: [] list photo_ids correct : [-3656800793, -3656791909, -3656800808, -3656791899, -3656800744, -3656800769, -3656800775, -3656800777, -3656800794, -3656800789, -3656800806, -3656800845, -3656800834, -3656791895, -3656791896, -3656791903, -3656800741, -3656800790, -3656800797, -3656800805, -3656791847, -3656800779, -3656800772, -3657192330, -3656800774, -3656800783, -3656800781, -3656800776, -3656800778, -3656800770, -3656800784, -3656800787, -3656800801, -3656800791, -3656800785, -3656800807, -3656800815, -3656917417, -3656800825, -3656800824, -3656800829, -3656800823, -3656800840, -3656791869, -3656791900, -3656800721, -3656800786, -3656800803, -3656800800, -3656800796, -3656800809, -3657192334, -3656791844, -3656791894, -3656800724, -3656800728, -3656800725, -3656800720, -3656800718, -3656800727, -3656800748, -3656800749, -3656800743, -3656791842, -3656791867, -3656791859, -3656791861, -3656791904, -3660098419, -3656791906, -3656791893, -3656791902, -3656791901, -3656800838, -3656800839, -3656800831, -3656800837, -3656917420, -3656917421, -3656800842, -3656800846, -3656791840, -3656791863, -3656791841, -3656791862, -3656791865, -3657189462, -3656800747, -3656800782, -3656800799, -3656800813, -3656800830, -3656800841] number of photos to traite : 93 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10774 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10555 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.0514130592346191 time used to do the prediction : 0.26033616065979004 we don't save the descriptors for this thcl 2456 tagging for thcl : 2868 To do loadFromThcl(), then load ParamDescType : thcl2868 thcls : [{'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5288 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) To loadFromThcl() : net_5288 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10553 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_papier_nantes_300421 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_papier_nantes_300421 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_papier_nantes_300421 /data/models_weight/learn_papier_nantes_300421/caffemodel size_local : 44791983 size in s3 : 44791983 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:13 caffemodel already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/deploy.prototxt size_local : 17255 size in s3 : 17255 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:12 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:14 mean.npy already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/synset_words.txt size_local : 331 size in s3 : 331 create time local : 2021-08-09 05:56:00 create time in s3 : 2021-08-06 19:22:12 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_papier_nantes_300421/deploy.prototxt caffemodel_filename : /data/models_weight/learn_papier_nantes_300421/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10553 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.0968105792999268 time used to do the prediction : 0.17908859252929688 we don't save the descriptors for this thcl 2868 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 1.6689300537109375e-05 save missing photos in datou_result : time spend for datou_step_exec : 10.329367399215698 time spend to save output : 0.0006685256958007812 total time spend for step 3 : 10.330035924911499 step4:merge_mask_thcl_custom Fri Feb 7 16:56:24 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 164 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1334879932, 1334879871, 1334879795, 1334879706, 1334879642, 1334879629, 1334879557, 1334879533, 1334879425, 1334879352] Looping around the photos to save general results len do output : 10 /1334879932Didn't retrieve data .Didn't retrieve data . /1334879871Didn't retrieve data .Didn't retrieve data . /1334879795Didn't retrieve data .Didn't retrieve data . /1334879706Didn't retrieve data .Didn't retrieve data . /1334879642Didn't retrieve data .Didn't retrieve data . /1334879629Didn't retrieve data .Didn't retrieve data . /1334879557Didn't retrieve data .Didn't retrieve data . /1334879533Didn't retrieve data .Didn't retrieve data . /1334879425Didn't retrieve data .Didn't retrieve data . /1334879352Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879932', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879871', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879795', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879706', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879642', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879629', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879557', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879533', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879425', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879352', None, None, None, None, None, '2565109') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.017742395401000977 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.06908535957336426 time spend to save output : 0.018274307250976562 total time spend for step 4 : 0.08735966682434082 step5:rle_unique_nms_with_priority Fri Feb 7 16:56:24 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 12 nb_hashtags : 4 time to prepare the origin masks : 7.415415287017822 time for calcul the mask position with numpy : 0.685981273651123 nb_pixel_total : 6530231 time to create 1 rle with new method : 0.4956364631652832 time for calcul the mask position with numpy : 0.02292656898498535 nb_pixel_total : 6339 time to create 1 rle with old method : 0.00720667839050293 time for calcul the mask position with numpy : 0.023040771484375 nb_pixel_total : 146006 time to create 1 rle with old method : 0.15805530548095703 time for calcul the mask position with numpy : 0.02285146713256836 nb_pixel_total : 19068 time to create 1 rle with old method : 0.020766258239746094 time for calcul the mask position with numpy : 0.021602153778076172 nb_pixel_total : 3404 time to create 1 rle with old method : 0.0039904117584228516 time for calcul the mask position with numpy : 0.02864384651184082 nb_pixel_total : 16076 time to create 1 rle with old method : 0.025209903717041016 time for calcul the mask position with numpy : 0.02318429946899414 nb_pixel_total : 12246 time to create 1 rle with old method : 0.017755985260009766 time for calcul the mask position with numpy : 0.0320124626159668 nb_pixel_total : 136552 time to create 1 rle with old method : 0.14635682106018066 time for calcul the mask position with numpy : 0.03771162033081055 nb_pixel_total : 15456 time to create 1 rle with old method : 0.0187225341796875 time for calcul the mask position with numpy : 0.03654336929321289 nb_pixel_total : 9383 time to create 1 rle with old method : 0.010381937026977539 time for calcul the mask position with numpy : 0.03547215461730957 nb_pixel_total : 189 time to create 1 rle with old method : 0.00038933753967285156 time for calcul the mask position with numpy : 0.03465080261230469 nb_pixel_total : 35894 time to create 1 rle with old method : 0.04590129852294922 time for calcul the mask position with numpy : 0.040967702865600586 nb_pixel_total : 119396 time to create 1 rle with old method : 0.13885068893432617 create new chi : 2.174705743789673 time to delete rle : 0.018206119537353516 batch 1 Loaded 14 chid ids of type : 3726 Number RLEs to save : 8211 TO DO : save crop sub photo not yet done ! save time : 0.5703177452087402 nb_obj : 11 nb_hashtags : 4 time to prepare the origin masks : 5.982203245162964 time for calcul the mask position with numpy : 0.6495797634124756 nb_pixel_total : 6357046 time to create 1 rle with new method : 0.445955753326416 time for calcul the mask position with numpy : 0.02111959457397461 nb_pixel_total : 2722 time to create 1 rle with old method : 0.003092527389526367 time for calcul the mask position with numpy : 0.02129983901977539 nb_pixel_total : 27809 time to create 1 rle with old method : 0.029806852340698242 time for calcul the mask position with numpy : 0.026986122131347656 nb_pixel_total : 196342 time to create 1 rle with new method : 0.430466890335083 time for calcul the mask position with numpy : 0.03639054298400879 nb_pixel_total : 7074 time to create 1 rle with old method : 0.00813913345336914 time for calcul the mask position with numpy : 0.03684115409851074 nb_pixel_total : 9120 time to create 1 rle with old method : 0.010819673538208008 time for calcul the mask position with numpy : 0.0355982780456543 nb_pixel_total : 5621 time to create 1 rle with old method : 0.006616115570068359 time for calcul the mask position with numpy : 0.0344698429107666 nb_pixel_total : 7626 time to create 1 rle with old method : 0.008201360702514648 time for calcul the mask position with numpy : 0.03703880310058594 nb_pixel_total : 289000 time to create 1 rle with new method : 0.37262773513793945 time for calcul the mask position with numpy : 0.03920745849609375 nb_pixel_total : 79418 time to create 1 rle with old method : 0.1053457260131836 time for calcul the mask position with numpy : 0.022716045379638672 nb_pixel_total : 48565 time to create 1 rle with old method : 0.05225825309753418 time for calcul the mask position with numpy : 0.022800683975219727 nb_pixel_total : 19897 time to create 1 rle with old method : 0.02187943458557129 create new chi : 2.5617053508758545 time to delete rle : 0.0013039112091064453 batch 1 Loaded 14 chid ids of type : 3726 Number RLEs to save : 8826 TO DO : save crop sub photo not yet done ! save time : 0.6478550434112549 nb_obj : 20 nb_hashtags : 6 time to prepare the origin masks : 12.56849479675293 time for calcul the mask position with numpy : 0.7576215267181396 nb_pixel_total : 6467020 time to create 1 rle with new method : 0.6647374629974365 time for calcul the mask position with numpy : 0.023497343063354492 nb_pixel_total : 3945 time to create 1 rle with old method : 0.00471186637878418 time for calcul the mask position with numpy : 0.02442169189453125 nb_pixel_total : 86931 time to create 1 rle with old method : 0.09156680107116699 time for calcul the mask position with numpy : 0.03779172897338867 nb_pixel_total : 2153 time to create 1 rle with old method : 0.0025186538696289062 time for calcul the mask position with numpy : 0.0336763858795166 nb_pixel_total : 8005 time to create 1 rle with old method : 0.00886988639831543 time for calcul the mask position with numpy : 0.0363922119140625 nb_pixel_total : 28617 time to create 1 rle with old method : 0.03005075454711914 time for calcul the mask position with numpy : 0.03602313995361328 nb_pixel_total : 2129 time to create 1 rle with old method : 0.002212047576904297 time for calcul the mask position with numpy : 0.03699517250061035 nb_pixel_total : 2351 time to create 1 rle with old method : 0.0026884078979492188 time for calcul the mask position with numpy : 0.03953385353088379 nb_pixel_total : 164087 time to create 1 rle with new method : 0.5272228717803955 time for calcul the mask position with numpy : 0.03621315956115723 nb_pixel_total : 35414 time to create 1 rle with old method : 0.03735756874084473 time for calcul the mask position with numpy : 0.03447246551513672 nb_pixel_total : 2985 time to create 1 rle with old method : 0.0032520294189453125 time for calcul the mask position with numpy : 0.034158945083618164 nb_pixel_total : 2057 time to create 1 rle with old method : 0.0023648738861083984 time for calcul the mask position with numpy : 0.03412318229675293 nb_pixel_total : 16167 time to create 1 rle with old method : 0.017355680465698242 time for calcul the mask position with numpy : 0.03470015525817871 nb_pixel_total : 34498 time to create 1 rle with old method : 0.03667163848876953 time for calcul the mask position with numpy : 0.03976297378540039 nb_pixel_total : 2051 time to create 1 rle with old method : 0.0023992061614990234 time for calcul the mask position with numpy : 0.03539919853210449 nb_pixel_total : 3916 time to create 1 rle with old method : 0.004502296447753906 time for calcul the mask position with numpy : 0.035036325454711914 nb_pixel_total : 104904 time to create 1 rle with old method : 0.12671804428100586 time for calcul the mask position with numpy : 0.032442569732666016 nb_pixel_total : 10876 time to create 1 rle with old method : 0.12775325775146484 time for calcul the mask position with numpy : 0.035608530044555664 nb_pixel_total : 2393 time to create 1 rle with old method : 0.0028493404388427734 time for calcul the mask position with numpy : 0.0339970588684082 nb_pixel_total : 57125 time to create 1 rle with old method : 0.06069445610046387 time for calcul the mask position with numpy : 0.032659053802490234 nb_pixel_total : 12616 time to create 1 rle with old method : 0.013946771621704102 create new chi : 3.2993555068969727 time to delete rle : 0.0014393329620361328 batch 1 Loaded 24 chid ids of type : 3726 Number RLEs to save : 10754 TO DO : save crop sub photo not yet done ! save time : 0.7170665264129639 nb_obj : 14 nb_hashtags : 4 time to prepare the origin masks : 6.689142942428589 time for calcul the mask position with numpy : 0.45760202407836914 nb_pixel_total : 6306432 time to create 1 rle with new method : 0.4782557487487793 time for calcul the mask position with numpy : 0.0218050479888916 nb_pixel_total : 3705 time to create 1 rle with old method : 0.00600123405456543 time for calcul the mask position with numpy : 0.022699356079101562 nb_pixel_total : 872 time to create 1 rle with old method : 0.0018260478973388672 time for calcul the mask position with numpy : 0.040584564208984375 nb_pixel_total : 57977 time to create 1 rle with old method : 0.08697986602783203 time for calcul the mask position with numpy : 0.0349118709564209 nb_pixel_total : 2319 time to create 1 rle with old method : 0.002626657485961914 time for calcul the mask position with numpy : 0.03405022621154785 nb_pixel_total : 5223 time to create 1 rle with old method : 0.005832195281982422 time for calcul the mask position with numpy : 0.03616523742675781 nb_pixel_total : 60556 time to create 1 rle with old method : 0.06462693214416504 time for calcul the mask position with numpy : 0.034354448318481445 nb_pixel_total : 28631 time to create 1 rle with old method : 0.03011322021484375 time for calcul the mask position with numpy : 0.03575444221496582 nb_pixel_total : 305301 time to create 1 rle with new method : 0.45817065238952637 time for calcul the mask position with numpy : 0.032117366790771484 nb_pixel_total : 3813 time to create 1 rle with old method : 0.004361629486083984 time for calcul the mask position with numpy : 0.02189922332763672 nb_pixel_total : 35975 time to create 1 rle with old method : 0.0380101203918457 time for calcul the mask position with numpy : 0.02208852767944336 nb_pixel_total : 25026 time to create 1 rle with old method : 0.026761770248413086 time for calcul the mask position with numpy : 0.02317357063293457 nb_pixel_total : 92947 time to create 1 rle with old method : 0.09878230094909668 time for calcul the mask position with numpy : 0.02291107177734375 nb_pixel_total : 118108 time to create 1 rle with old method : 0.1282970905303955 time for calcul the mask position with numpy : 0.024657726287841797 nb_pixel_total : 3355 time to create 1 rle with old method : 0.0038433074951171875 create new chi : 2.367316484451294 time to delete rle : 0.0009813308715820312 batch 1 Loaded 15 chid ids of type : 3726 Number RLEs to save : 8718 TO DO : save crop sub photo not yet done ! save time : 0.7442378997802734 nb_obj : 11 nb_hashtags : 4 time to prepare the origin masks : 5.728154420852661 time for calcul the mask position with numpy : 0.7685995101928711 nb_pixel_total : 6597211 time to create 1 rle with new method : 0.5990612506866455 time for calcul the mask position with numpy : 0.021918535232543945 nb_pixel_total : 19250 time to create 1 rle with old method : 0.01892995834350586 time for calcul the mask position with numpy : 0.021825790405273438 nb_pixel_total : 29645 time to create 1 rle with old method : 0.029646635055541992 time for calcul the mask position with numpy : 0.022526979446411133 nb_pixel_total : 54689 time to create 1 rle with old method : 0.05629539489746094 time for calcul the mask position with numpy : 0.02274632453918457 nb_pixel_total : 57222 time to create 1 rle with old method : 0.05933189392089844 time for calcul the mask position with numpy : 0.023555278778076172 nb_pixel_total : 8636 time to create 1 rle with old method : 0.009184598922729492 time for calcul the mask position with numpy : 0.02347731590270996 nb_pixel_total : 151408 time to create 1 rle with new method : 0.48279809951782227 time for calcul the mask position with numpy : 0.024283885955810547 nb_pixel_total : 26214 time to create 1 rle with old method : 0.028225421905517578 time for calcul the mask position with numpy : 0.024755001068115234 nb_pixel_total : 6006 time to create 1 rle with old method : 0.005991458892822266 time for calcul the mask position with numpy : 0.02523660659790039 nb_pixel_total : 59558 time to create 1 rle with old method : 0.06164979934692383 time for calcul the mask position with numpy : 0.024180173873901367 nb_pixel_total : 33700 time to create 1 rle with old method : 0.03478717803955078 time for calcul the mask position with numpy : 0.024714946746826172 nb_pixel_total : 6701 time to create 1 rle with old method : 0.007511615753173828 create new chi : 2.4978301525115967 time to delete rle : 0.0008618831634521484 batch 1 Loaded 13 chid ids of type : 3726 Number RLEs to save : 8626 TO DO : save crop sub photo not yet done ! save time : 0.6450588703155518 nb_obj : 16 nb_hashtags : 7 time to prepare the origin masks : 9.645121335983276 time for calcul the mask position with numpy : 0.4173707962036133 nb_pixel_total : 6196878 time to create 1 rle with new method : 0.4406261444091797 time for calcul the mask position with numpy : 0.025603771209716797 nb_pixel_total : 23667 time to create 1 rle with old method : 0.024890422821044922 time for calcul the mask position with numpy : 0.02458047866821289 nb_pixel_total : 19509 time to create 1 rle with old method : 0.02129364013671875 time for calcul the mask position with numpy : 0.02639484405517578 nb_pixel_total : 165321 time to create 1 rle with new method : 0.8635766506195068 time for calcul the mask position with numpy : 0.02472519874572754 nb_pixel_total : 61828 time to create 1 rle with old method : 0.0659186840057373 time for calcul the mask position with numpy : 0.02417469024658203 nb_pixel_total : 21133 time to create 1 rle with old method : 0.02247762680053711 time for calcul the mask position with numpy : 0.04188990592956543 nb_pixel_total : 126761 time to create 1 rle with old method : 0.1311795711517334 time for calcul the mask position with numpy : 0.03899097442626953 nb_pixel_total : 1675 time to create 1 rle with old method : 0.0019176006317138672 time for calcul the mask position with numpy : 0.04289555549621582 nb_pixel_total : 53569 time to create 1 rle with old method : 0.05515146255493164 time for calcul the mask position with numpy : 0.03894805908203125 nb_pixel_total : 38457 time to create 1 rle with old method : 0.039943695068359375 time for calcul the mask position with numpy : 0.042891740798950195 nb_pixel_total : 6980 time to create 1 rle with old method : 0.008724451065063477 time for calcul the mask position with numpy : 0.04560232162475586 nb_pixel_total : 4629 time to create 1 rle with old method : 0.004951000213623047 time for calcul the mask position with numpy : 0.03778696060180664 nb_pixel_total : 24657 time to create 1 rle with old method : 0.025648832321166992 time for calcul the mask position with numpy : 0.03725790977478027 nb_pixel_total : 52635 time to create 1 rle with old method : 0.06026721000671387 time for calcul the mask position with numpy : 0.03209257125854492 nb_pixel_total : 106394 time to create 1 rle with old method : 0.11537528038024902 time for calcul the mask position with numpy : 0.03262209892272949 nb_pixel_total : 79510 time to create 1 rle with old method : 0.09006357192993164 time for calcul the mask position with numpy : 0.05252885818481445 nb_pixel_total : 66637 time to create 1 rle with old method : 0.07446837425231934 create new chi : 3.1163101196289062 time to delete rle : 0.001984834671020508 batch 1 Loaded 17 chid ids of type : 3726 Number RLEs to save : 13289 TO DO : save crop sub photo not yet done ! save time : 0.9464669227600098 nb_obj : 19 nb_hashtags : 6 time to prepare the origin masks : 9.347254037857056 time for calcul the mask position with numpy : 0.3656795024871826 nb_pixel_total : 6289954 time to create 1 rle with new method : 0.6458699703216553 time for calcul the mask position with numpy : 0.022314071655273438 nb_pixel_total : 35749 time to create 1 rle with old method : 0.03848886489868164 time for calcul the mask position with numpy : 0.021664857864379883 nb_pixel_total : 32674 time to create 1 rle with old method : 0.034742116928100586 time for calcul the mask position with numpy : 0.026700258255004883 nb_pixel_total : 140 time to create 1 rle with old method : 0.0003020763397216797 time for calcul the mask position with numpy : 0.03570222854614258 nb_pixel_total : 11991 time to create 1 rle with old method : 0.012870550155639648 time for calcul the mask position with numpy : 0.03766298294067383 nb_pixel_total : 23220 time to create 1 rle with old method : 0.02514481544494629 time for calcul the mask position with numpy : 0.03416037559509277 nb_pixel_total : 34632 time to create 1 rle with old method : 0.054265737533569336 time for calcul the mask position with numpy : 0.0368647575378418 nb_pixel_total : 5052 time to create 1 rle with old method : 0.005726814270019531 time for calcul the mask position with numpy : 0.03444409370422363 nb_pixel_total : 13085 time to create 1 rle with old method : 0.014672517776489258 time for calcul the mask position with numpy : 0.03479814529418945 nb_pixel_total : 6657 time to create 1 rle with old method : 0.007550477981567383 time for calcul the mask position with numpy : 0.025915861129760742 nb_pixel_total : 2505 time to create 1 rle with old method : 0.0028662681579589844 time for calcul the mask position with numpy : 0.0266721248626709 nb_pixel_total : 1099 time to create 1 rle with old method : 0.001584768295288086 time for calcul the mask position with numpy : 0.026621103286743164 nb_pixel_total : 4382 time to create 1 rle with old method : 0.005358457565307617 time for calcul the mask position with numpy : 0.03500509262084961 nb_pixel_total : 1744 time to create 1 rle with old method : 0.0020415782928466797 time for calcul the mask position with numpy : 0.03647136688232422 nb_pixel_total : 51180 time to create 1 rle with old method : 0.05438494682312012 time for calcul the mask position with numpy : 0.03385114669799805 nb_pixel_total : 9849 time to create 1 rle with old method : 0.010532379150390625 time for calcul the mask position with numpy : 0.03423500061035156 nb_pixel_total : 37973 time to create 1 rle with old method : 0.0405271053314209 time for calcul the mask position with numpy : 0.04232025146484375 nb_pixel_total : 366745 time to create 1 rle with new method : 0.45671582221984863 time for calcul the mask position with numpy : 0.038457393646240234 nb_pixel_total : 66858 time to create 1 rle with old method : 0.07535505294799805 time for calcul the mask position with numpy : 0.037741899490356445 nb_pixel_total : 54751 time to create 1 rle with old method : 0.06180548667907715 create new chi : 2.602576494216919 time to delete rle : 0.0010488033294677734 batch 1 Loaded 21 chid ids of type : 3726 Number RLEs to save : 10154 TO DO : save crop sub photo not yet done ! save time : 0.6880078315734863 nb_obj : 11 nb_hashtags : 7 time to prepare the origin masks : 6.0871806144714355 time for calcul the mask position with numpy : 0.4759175777435303 nb_pixel_total : 6673300 time to create 1 rle with new method : 0.42383837699890137 time for calcul the mask position with numpy : 0.024164915084838867 nb_pixel_total : 24571 time to create 1 rle with old method : 0.025989055633544922 time for calcul the mask position with numpy : 0.022693872451782227 nb_pixel_total : 25281 time to create 1 rle with old method : 0.02595829963684082 time for calcul the mask position with numpy : 0.024358510971069336 nb_pixel_total : 92087 time to create 1 rle with old method : 0.09330296516418457 time for calcul the mask position with numpy : 0.02325749397277832 nb_pixel_total : 29384 time to create 1 rle with old method : 0.030236005783081055 time for calcul the mask position with numpy : 0.022154569625854492 nb_pixel_total : 1169 time to create 1 rle with old method : 0.001348733901977539 time for calcul the mask position with numpy : 0.022832155227661133 nb_pixel_total : 60634 time to create 1 rle with old method : 0.06135058403015137 time for calcul the mask position with numpy : 0.023696422576904297 nb_pixel_total : 14724 time to create 1 rle with old method : 0.014768838882446289 time for calcul the mask position with numpy : 0.023919343948364258 nb_pixel_total : 50928 time to create 1 rle with old method : 0.056087493896484375 time for calcul the mask position with numpy : 0.026859521865844727 nb_pixel_total : 20363 time to create 1 rle with old method : 0.02151179313659668 time for calcul the mask position with numpy : 0.026627779006958008 nb_pixel_total : 5945 time to create 1 rle with old method : 0.0068628787994384766 time for calcul the mask position with numpy : 0.02625417709350586 nb_pixel_total : 51854 time to create 1 rle with old method : 0.0529935359954834 create new chi : 1.6084620952606201 time to delete rle : 0.0009047985076904297 batch 1 Loaded 13 chid ids of type : 3726 Number RLEs to save : 7388 TO DO : save crop sub photo not yet done ! save time : 0.49118947982788086 nb_obj : 12 nb_hashtags : 4 time to prepare the origin masks : 5.667290925979614 time for calcul the mask position with numpy : 0.5078792572021484 nb_pixel_total : 6787696 time to create 1 rle with new method : 0.6003873348236084 time for calcul the mask position with numpy : 0.020583152770996094 nb_pixel_total : 493 time to create 1 rle with old method : 0.0006892681121826172 time for calcul the mask position with numpy : 0.022474050521850586 nb_pixel_total : 127176 time to create 1 rle with old method : 0.13380908966064453 time for calcul the mask position with numpy : 0.02076888084411621 nb_pixel_total : 25885 time to create 1 rle with old method : 0.02704596519470215 time for calcul the mask position with numpy : 0.020641088485717773 nb_pixel_total : 9034 time to create 1 rle with old method : 0.00931859016418457 time for calcul the mask position with numpy : 0.021257400512695312 nb_pixel_total : 166 time to create 1 rle with old method : 0.000286102294921875 time for calcul the mask position with numpy : 0.021440505981445312 nb_pixel_total : 9038 time to create 1 rle with old method : 0.009080886840820312 time for calcul the mask position with numpy : 0.020436763763427734 nb_pixel_total : 37032 time to create 1 rle with old method : 0.04023933410644531 time for calcul the mask position with numpy : 0.02128314971923828 nb_pixel_total : 2178 time to create 1 rle with old method : 0.002389192581176758 time for calcul the mask position with numpy : 0.021251916885375977 nb_pixel_total : 24198 time to create 1 rle with old method : 0.025452375411987305 time for calcul the mask position with numpy : 0.022404909133911133 nb_pixel_total : 5696 time to create 1 rle with old method : 0.006364583969116211 time for calcul the mask position with numpy : 0.022337913513183594 nb_pixel_total : 12336 time to create 1 rle with old method : 0.013103008270263672 time for calcul the mask position with numpy : 0.02489471435546875 nb_pixel_total : 9312 time to create 1 rle with old method : 0.009995222091674805 create new chi : 1.6920766830444336 time to delete rle : 0.000701904296875 batch 1 Loaded 14 chid ids of type : 3726 Number RLEs to save : 7074 TO DO : save crop sub photo not yet done ! save time : 0.4923868179321289 nb_obj : 17 nb_hashtags : 7 time to prepare the origin masks : 8.642398357391357 time for calcul the mask position with numpy : 0.3775062561035156 nb_pixel_total : 5852151 time to create 1 rle with new method : 0.42128777503967285 time for calcul the mask position with numpy : 0.021841049194335938 nb_pixel_total : 51419 time to create 1 rle with old method : 0.056519269943237305 time for calcul the mask position with numpy : 0.023088455200195312 nb_pixel_total : 1588 time to create 1 rle with old method : 0.002161741256713867 time for calcul the mask position with numpy : 0.02127981185913086 nb_pixel_total : 17410 time to create 1 rle with old method : 0.019318580627441406 time for calcul the mask position with numpy : 0.021270036697387695 nb_pixel_total : 19049 time to create 1 rle with old method : 0.020679235458374023 time for calcul the mask position with numpy : 0.022648096084594727 nb_pixel_total : 77544 time to create 1 rle with old method : 0.08592367172241211 time for calcul the mask position with numpy : 0.02478790283203125 nb_pixel_total : 316146 time to create 1 rle with new method : 0.4229423999786377 time for calcul the mask position with numpy : 0.022661924362182617 nb_pixel_total : 51106 time to create 1 rle with old method : 0.057711124420166016 time for calcul the mask position with numpy : 0.023056745529174805 nb_pixel_total : 188686 time to create 1 rle with new method : 0.44797301292419434 time for calcul the mask position with numpy : 0.021906614303588867 nb_pixel_total : 24989 time to create 1 rle with old method : 0.0322270393371582 time for calcul the mask position with numpy : 0.02262091636657715 nb_pixel_total : 62851 time to create 1 rle with old method : 0.0661165714263916 time for calcul the mask position with numpy : 0.023430585861206055 nb_pixel_total : 242148 time to create 1 rle with new method : 0.33594775199890137 time for calcul the mask position with numpy : 0.02050948143005371 nb_pixel_total : 18732 time to create 1 rle with old method : 0.019484281539916992 time for calcul the mask position with numpy : 0.02171635627746582 nb_pixel_total : 35191 time to create 1 rle with old method : 0.036916255950927734 time for calcul the mask position with numpy : 0.019922256469726562 nb_pixel_total : 4827 time to create 1 rle with old method : 0.0052683353424072266 time for calcul the mask position with numpy : 0.02046370506286621 nb_pixel_total : 5374 time to create 1 rle with old method : 0.0058133602142333984 time for calcul the mask position with numpy : 0.02049851417541504 nb_pixel_total : 22898 time to create 1 rle with old method : 0.025241851806640625 time for calcul the mask position with numpy : 0.02126288414001465 nb_pixel_total : 58131 time to create 1 rle with old method : 0.06084132194519043 create new chi : 2.9874136447906494 time to delete rle : 0.0013430118560791016 batch 1 Loaded 20 chid ids of type : 3726 Number RLEs to save : 14511 TO DO : save crop sub photo not yet done ! save time : 0.9782345294952393 map_output_result : {1334879932: (0.0, 'Should be the crop_list due to order', 0.0), 1334879871: (0.0, 'Should be the crop_list due to order', 0.0), 1334879795: (0.0, 'Should be the crop_list due to order', 0.0), 1334879706: (0.0, 'Should be the crop_list due to order', 0.0), 1334879642: (0.0, 'Should be the crop_list due to order', 0.0), 1334879629: (0.0, 'Should be the crop_list due to order', 0.0), 1334879557: (0.0, 'Should be the crop_list due to order', 0.0), 1334879533: (0.0, 'Should be the crop_list due to order', 0.0), 1334879425: (0.0, 'Should be the crop_list due to order', 0.0), 1334879352: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1334879932, 1334879871, 1334879795, 1334879706, 1334879642, 1334879629, 1334879557, 1334879533, 1334879425, 1334879352] Looping around the photos to save general results len do output : 10 /1334879932.Didn't retrieve data . /1334879871.Didn't retrieve data . /1334879795.Didn't retrieve data . /1334879706.Didn't retrieve data . /1334879642.Didn't retrieve data . /1334879629.Didn't retrieve data . /1334879557.Didn't retrieve data . /1334879533.Didn't retrieve data . /1334879425.Didn't retrieve data . /1334879352.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879932', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879871', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879795', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879706', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879642', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879629', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879557', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879533', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879425', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879352', None, None, None, None, None, '2565109') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.017486095428466797 save_final save missing photos in datou_result : time spend for datou_step_exec : 110.57323694229126 time spend to save output : 0.017945289611816406 total time spend for step 5 : 110.59118223190308 step6:crop_condition Fri Feb 7 16:58:14 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 165 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 4869462 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738943897_284429 we have uploaded 4 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.8513121604919434 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 11 About to insert : list_path_to_insert length 11 new photo from crops ! About to upload 11 photos upload in portfolio : 4869462 init cache_photo without model_param we have 11 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738943906_284429 we have uploaded 11 photos in the portfolio 4869462 time of upload the photos Elapsed time : 3.2768361568450928 we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 7 new photo from crops ! About to upload 7 photos upload in portfolio : 4869462 init cache_photo without model_param we have 7 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738943912_284429 we have uploaded 7 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.9432170391082764 we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 4869462 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738943916_284429 we have uploaded 2 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.753472089767456 we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 6 new photo from crops ! About to upload 6 photos upload in portfolio : 4869462 init cache_photo without model_param we have 6 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738943921_284429 we have uploaded 6 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.7853426933288574 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738943924_284429 we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.0339422225952148 we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1334879932, 1334879871, 1334879795, 1334879706, 1334879642, 1334879629, 1334879557, 1334879533, 1334879425, 1334879352] Looping around the photos to save general results len do output : 35 /1335553739Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553741Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553743Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553745Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553853Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553854Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553855Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553856Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553857Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553858Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553859Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553860Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553861Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553862Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553863Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553877Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553878Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553879Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553880Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553881Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553882Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335124402Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553883Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553901Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553902Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553907Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553909Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553910Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335124447Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553911Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553912Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553913Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553915Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553916Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335553917Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879932', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879871', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879795', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879706', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879642', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879629', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879557', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879533', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879425', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879352', None, None, None, None, None, '2565109') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 115 time used for this insertion : 0.028554439544677734 save_final save missing photos in datou_result : time spend for datou_step_exec : 31.644208192825317 time spend to save output : 0.029645919799804688 total time spend for step 6 : 31.673854112625122 step7:ventilate_hashtags_in_portfolio Fri Feb 7 16:58:46 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 20341531 get user id for portfolio 20341531 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20341531 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Teint_Dans_La_Masse','kraft','mal_croppe','environnement','Carton_gris','autre_refus','papier','Carton_brun','flou','cartonnette','plastique','metal')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20341531 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Teint_Dans_La_Masse','kraft','mal_croppe','environnement','Carton_gris','autre_refus','papier','Carton_brun','flou','cartonnette','plastique','metal')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/20341721,20341722,20341723,20341724,20341725,20341726,20341727,20341728,20341729,20341730,20341731,20341732?tags=Carton_gris,flou,plastique,mal_croppe,Carton_brun,metal,kraft,papier,environnement,autre_refus,Teint_Dans_La_Masse,cartonnette Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1334879932, 1334879871, 1334879795, 1334879706, 1334879642, 1334879629, 1334879557, 1334879533, 1334879425, 1334879352] Looping around the photos to save general results len do output : 1 /20341531. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879932', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879871', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879795', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879706', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879642', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879629', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879557', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879533', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879425', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879352', None, None, None, None, None, '2565109') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.016020774841308594 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.6562023162841797 time spend to save output : 0.016309261322021484 total time spend for step 7 : 1.6725115776062012 step8:final Fri Feb 7 16:58:48 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1334879932: ('0.08004729661136349',), 1334879871: ('0.08004729661136349',), 1334879795: ('0.08004729661136349',), 1334879706: ('0.08004729661136349',), 1334879642: ('0.08004729661136349',), 1334879629: ('0.08004729661136349',), 1334879557: ('0.08004729661136349',), 1334879533: ('0.08004729661136349',), 1334879425: ('0.08004729661136349',), 1334879352: ('0.08004729661136349',)} new output for save of step final : {1334879932: ('0.08004729661136349',), 1334879871: ('0.08004729661136349',), 1334879795: ('0.08004729661136349',), 1334879706: ('0.08004729661136349',), 1334879642: ('0.08004729661136349',), 1334879629: ('0.08004729661136349',), 1334879557: ('0.08004729661136349',), 1334879533: ('0.08004729661136349',), 1334879425: ('0.08004729661136349',), 1334879352: ('0.08004729661136349',)} [1334879932, 1334879871, 1334879795, 1334879706, 1334879642, 1334879629, 1334879557, 1334879533, 1334879425, 1334879352] Looping around the photos to save general results len do output : 10 /1334879932.Didn't retrieve data . /1334879871.Didn't retrieve data . /1334879795.Didn't retrieve data . /1334879706.Didn't retrieve data . /1334879642.Didn't retrieve data . /1334879629.Didn't retrieve data . /1334879557.Didn't retrieve data . /1334879533.Didn't retrieve data . /1334879425.Didn't retrieve data . /1334879352.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879932', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879871', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879795', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879706', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879642', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879629', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879557', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879533', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879425', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879352', None, None, None, None, None, '2565109') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.021368026733398438 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.1330583095550537 time spend to save output : 0.021938562393188477 total time spend for step 8 : 0.1549968719482422 step9:velours_tree Fri Feb 7 16:58:48 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.04567074775695801 time spend to save output : 5.745887756347656e-05 total time spend for step 9 : 0.045728206634521484 step10:send_mail_cod Fri Feb 7 16:58:48 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P20341531_07-02-2025_16_58_48.pdf 20341722 imagette203417221738943928 20341723 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette203417231738943928 20341724 imagette203417241738943929 20341726 imagette203417261738943929 20341727 change filename to text .imagette203417271738943929 20341730 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette203417301738943929 20341732 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette203417321738943930 20341721 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette203417211738943931 20341725 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette203417251738943933 20341731 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette203417311738943934 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20341531 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20341721,20341722,20341723,20341724,20341725,20341726,20341727,20341728,20341729,20341730,20341731,20341732?tags=Carton_gris,flou,plastique,mal_croppe,Carton_brun,metal,kraft,papier,environnement,autre_refus,Teint_Dans_La_Masse,cartonnette your option no_mail is active, we will not send the real mail to your client args[1334879932] : ('0.08004729661136349',) no score found for photo 1334879932 apple ('0.08004729661136349',) We are sending mail with results at report@fotonower.com args[1334879871] : ('0.08004729661136349',) no score found for photo 1334879871 apple ('0.08004729661136349',) We are sending mail with results at report@fotonower.com args[1334879795] : ('0.08004729661136349',) no score found for photo 1334879795 apple ('0.08004729661136349',) We are sending mail with results at report@fotonower.com args[1334879706] : ('0.08004729661136349',) no score found for photo 1334879706 apple ('0.08004729661136349',) We are sending mail with results at report@fotonower.com args[1334879642] : ('0.08004729661136349',) no score found for photo 1334879642 apple ('0.08004729661136349',) We are sending mail with results at report@fotonower.com args[1334879629] : ('0.08004729661136349',) no score found for photo 1334879629 apple ('0.08004729661136349',) We are sending mail with results at report@fotonower.com args[1334879557] : ('0.08004729661136349',) no score found for photo 1334879557 apple ('0.08004729661136349',) We are sending mail with results at report@fotonower.com args[1334879533] : ('0.08004729661136349',) no score found for photo 1334879533 apple ('0.08004729661136349',) We are sending mail with results at report@fotonower.com args[1334879425] : ('0.08004729661136349',) no score found for photo 1334879425 apple ('0.08004729661136349',) We are sending mail with results at report@fotonower.com args[1334879352] : ('0.08004729661136349',) no score found for photo 1334879352 apple ('0.08004729661136349',) We are sending mail with results at report@fotonower.com refus_total : 0.08004729661136349 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20341531 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1334879932,1335072435,1335072433,1335072422,1335072409,1334880241,1334879871,1334879795,1334879642,1334879629,1334879533,1334879425,1334879352,1334879303,1334879249,1334879706,1334880227,1334880231,1334879557,1335072399) Found this number of photos: 20 begin to download photo : 1334879932 begin to download photo : 1334880241 begin to download photo : 1334879533 begin to download photo : 1334879706 download finish for photo 1334879533 begin to download photo : 1334879425 download finish for photo 1334879932 begin to download photo : 1335072435 download finish for photo 1334879706 begin to download photo : 1334880227 download finish for photo 1334880241 begin to download photo : 1334879871 download finish for photo 1334879425 begin to download photo : 1334879352 download finish for photo 1334880227 begin to download photo : 1334880231 download finish for photo 1334879871 begin to download photo : 1334879795 download finish for photo 1335072435 begin to download photo : 1335072433 download finish for photo 1334879352 begin to download photo : 1334879303 download finish for photo 1334879795 begin to download photo : 1334879642 download finish for photo 1335072433 begin to download photo : 1335072422 download finish for photo 1334880231 begin to download photo : 1334879557 download finish for photo 1334879303 begin to download photo : 1334879249 download finish for photo 1334879557 begin to download photo : 1335072399 download finish for photo 1334879642 begin to download photo : 1334879629 download finish for photo 1335072422 begin to download photo : 1335072409 download finish for photo 1335072399 download finish for photo 1334879629 download finish for photo 1334879249 download finish for photo 1335072409 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20341531_07-02-2025_16_58_48.pdf results_Auto_P20341531_07-02-2025_16_58_48.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20341531_07-02-2025_16_58_48.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3459','20341531','results_Auto_P20341531_07-02-2025_16_58_48.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20341531_07-02-2025_16_58_48.pdf','pdf','','1.18','0.08004729661136349') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1334879932, 1334879871, 1334879795, 1334879706, 1334879642, 1334879629, 1334879557, 1334879533, 1334879425, 1334879352] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879932', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879871', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879795', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879706', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879642', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879629', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879557', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879533', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879425', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879352', None, None, None, None, None, '2565109') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.018919706344604492 save_final save missing photos in datou_result : time spend for datou_step_exec : 11.984576225280762 time spend to save output : 0.01926112174987793 total time spend for step 10 : 12.00383734703064 step11:split_time_score Fri Feb 7 16:59:00 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('10', 22),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 05022025 20341531 Nombre de photos uploadées : 22 / 23040 (0%) 05022025 20341531 Nombre de photos taguées (types de déchets): 0 / 22 (0%) 05022025 20341531 Nombre de photos taguées (volume) : 0 / 22 (0%) elapsed_time : load_data_split_time_score 2.6226043701171875e-06 elapsed_time : order_list_meta_photo_and_scores 8.58306884765625e-06 ?????????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.004751682281494141 elapsed_time : insert_dashboard_record_day_entry 0.028519392013549805 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20341524 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20341525 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20341526 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20341527 order by id desc limit 1 Qualite : 0.1599211058721707 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20277504_07-02-2025_01_04_17.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277504 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20277504 AND mptpi.`type`=3327 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20341528 order by id desc limit 1 Qualite : 0.05380256531138403 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20286197_06-02-2025_17_47_49.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20286197 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20286197 AND mptpi.`type`=4207 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20341529 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20341530 order by id desc limit 1 Qualite : 0.0797439683530309 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20341531_07-02-2025_16_58_48.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20341531 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20341531 AND mptpi.`type`=3726 To do Qualite : 0.21787861122458246 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20286226_06-02-2025_05_57_11.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20286226 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20286226 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277510 order by id desc limit 1 Qualite : 0.2311395498592955 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20277511_06-02-2025_02_48_24.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277511 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20277511 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20341532 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20341545 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20341558 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20341559 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20341560 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20341561 order by id desc limit 1 Qualite : 0.2571360868566177 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20286232 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20286232 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20341574 order by id desc limit 1 Qualite : 0.0853594633280579 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20286235_06-02-2025_08_55_57.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20286235 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20286235 AND mptpi.`type`=4207 To do Qualite : 0.04168433375162447 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20286236_06-02-2025_06_24_11.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20286236 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20286236 AND mptpi.`type`=4207 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20286237 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20341587 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20341588 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277522 order by id desc limit 1 Qualite : 0.1770747284253208 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20277523_06-02-2025_01_29_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277523 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20277523 AND mptpi.`type`=3594 To do Qualite : 0.07745378844214007 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20286240_06-02-2025_13_48_51.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20286240 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20286240 AND mptpi.`type`=4207 To do Qualite : 0.041799027803780364 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20286241_06-02-2025_05_41_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20286241 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20286241 AND mptpi.`type`=4207 To do Qualite : 0.04323377929993904 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20286242_06-02-2025_12_34_35.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20286242 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20286242 AND mptpi.`type`=4207 To do Qualite : 0.07058148054050647 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20277526_06-02-2025_02_22_35.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277526 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20277526 AND mptpi.`type`=4207 To do Qualite : 0.06769655641860853 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20277527_06-02-2025_00_59_31.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277527 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20277527 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277528 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277529 order by id desc limit 1 Qualite : 0.21480488134682896 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20277530_06-02-2025_01_27_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277530 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20277530 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277531 order by id desc limit 1 Qualite : 0.1712722669489088 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20286243_07-02-2025_00_32_04.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20286243 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20286243 AND mptpi.`type`=3327 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277533 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277534 order by id desc limit 1 Qualite : 0.09479940201243531 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20277535_06-02-2025_03_28_51.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277535 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20277535 AND mptpi.`type`=4207 To do Qualite : 0.05092112302991469 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20277548_06-02-2025_02_23_02.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277548 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20277548 AND mptpi.`type`=4207 To do Qualite : 0.180145604972313 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20277549_06-02-2025_00_58_40.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277549 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20277549 AND mptpi.`type`=3594 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20277550_06-02-2025_01_20_52.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277550 order by id desc limit 1 Qualite : 0.09609489658538523 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20277551_06-02-2025_02_21_21.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277551 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20277551 AND mptpi.`type`=3726 To do Qualite : 0.0744879671394418 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20286244_06-02-2025_06_04_52.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20286244 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20286244 AND mptpi.`type`=4207 To do Qualite : 0.2163261936236685 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20286245_06-02-2025_06_47_47.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20286245 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20286245 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20286246 order by id desc limit 1 NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'05022025': {'nb_upload': 22, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1334879932, 1334879871, 1334879795, 1334879706, 1334879642, 1334879629, 1334879557, 1334879533, 1334879425, 1334879352] Looping around the photos to save general results len do output : 1 /20341531Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879932', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879871', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879795', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879706', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879642', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879629', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879557', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879533', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879425', None, None, None, None, None, '2565109') ('3459', None, None, None, None, None, None, None, '2565109') ('3459', None, '1334879352', None, None, None, None, None, '2565109') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.015003681182861328 save_final save missing photos in datou_result : time spend for datou_step_exec : 21.88326144218445 time spend to save output : 0.015414237976074219 total time spend for step 11 : 21.898675680160522 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 147.69user 100.66system 4:56.43elapsed 83%CPU (0avgtext+0avgdata 3947080maxresident)k 2727640inputs+161728outputs (9695major+11161873minor)pagefaults 0swaps