python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 3459 -l 10 ' -s datou_current_3459 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 1838997 load datou : 3459 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) load thcls load THCL from format json or kwargs add thcl : 2456 in CacheModelConfig load THCL from format json or kwargs add thcl : 2868 in CacheModelConfig load THCL from format json or kwargs add thcl : 2896 in CacheModelConfig load pdts add pdt : 4999 in CacheModelConfig add pdt : 5288 in CacheModelConfig add pdt : 5309 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3459, datou_cur_ids : ['2557221'] with mtr_portfolio_ids : ['20277551'] and first list_photo_ids : [] new path : /proc/1838997/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, thcl, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 10 ; length of list_pids : 10 ; length of list_args : 10 time to download the photos : 2.8139188289642334 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Thu Feb 6 02:16:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10998 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-06 02:16:34.359508: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-06 02:16:34.387215: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-06 02:16:34.389378: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f0124000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-06 02:16:34.389415: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-06 02:16:34.393741: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-06 02:16:34.709744: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x3f497870 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-06 02:16:34.709795: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-06 02:16:34.711111: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-06 02:16:34.711585: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-06 02:16:34.714662: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-06 02:16:34.717732: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-06 02:16:34.718252: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-06 02:16:34.721090: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-06 02:16:34.722198: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-06 02:16:34.727052: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-06 02:16:34.728625: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-06 02:16:34.728707: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-06 02:16:34.729524: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-06 02:16:34.729540: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-06 02:16:34.729550: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-06 02:16:34.730986: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10193 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-06 02:16:34.996548: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-06 02:16:34.996651: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-06 02:16:34.996700: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-06 02:16:34.996732: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-06 02:16:34.996761: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-06 02:16:34.996789: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-06 02:16:34.996817: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-06 02:16:34.996846: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-06 02:16:34.999104: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-06 02:16:35.000481: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-06 02:16:35.000509: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-06 02:16:35.000524: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-06 02:16:35.000539: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-06 02:16:35.000553: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-06 02:16:35.000567: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-06 02:16:35.000581: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-06 02:16:35.000596: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-06 02:16:35.001840: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-06 02:16:35.001865: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-06 02:16:35.001873: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-06 02:16:35.001880: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-06 02:16:35.003194: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10193 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-06 02:16:42.600462: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-06 02:16:42.785404: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 10 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 17 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 17 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 18 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 23 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 27 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 28 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 24 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 16 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 5 Detection mask done ! Trying to reset tf kernel 1839533 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5706 tf kernel not reseted sub process len(results) : 10 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 10 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10998 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.00015664100646972656 nb_pixel_total : 1785 time to create 1 rle with old method : 0.002218484878540039 length of segment : 52 time for calcul the mask position with numpy : 0.0003120899200439453 nb_pixel_total : 9370 time to create 1 rle with old method : 0.01073145866394043 length of segment : 153 time for calcul the mask position with numpy : 0.0008792877197265625 nb_pixel_total : 35351 time to create 1 rle with old method : 0.03796267509460449 length of segment : 281 time for calcul the mask position with numpy : 0.0001609325408935547 nb_pixel_total : 6143 time to create 1 rle with old method : 0.006919145584106445 length of segment : 109 time for calcul the mask position with numpy : 0.0020248889923095703 nb_pixel_total : 79391 time to create 1 rle with old method : 0.08242678642272949 length of segment : 466 time for calcul the mask position with numpy : 0.0029675960540771484 nb_pixel_total : 110716 time to create 1 rle with old method : 0.12273502349853516 length of segment : 471 time for calcul the mask position with numpy : 0.0003235340118408203 nb_pixel_total : 13287 time to create 1 rle with old method : 0.016109943389892578 length of segment : 132 time for calcul the mask position with numpy : 0.00017023086547851562 nb_pixel_total : 3680 time to create 1 rle with old method : 0.004728794097900391 length of segment : 104 time for calcul the mask position with numpy : 7.390975952148438e-05 nb_pixel_total : 1471 time to create 1 rle with old method : 0.0018877983093261719 length of segment : 45 time for calcul the mask position with numpy : 0.005431413650512695 nb_pixel_total : 288835 time to create 1 rle with new method : 0.021435976028442383 length of segment : 626 time for calcul the mask position with numpy : 0.00013065338134765625 nb_pixel_total : 2677 time to create 1 rle with old method : 0.0032176971435546875 length of segment : 81 time for calcul the mask position with numpy : 0.00013685226440429688 nb_pixel_total : 6333 time to create 1 rle with old method : 0.0072176456451416016 length of segment : 87 time for calcul the mask position with numpy : 5.459785461425781e-05 nb_pixel_total : 1012 time to create 1 rle with old method : 0.0013356208801269531 length of segment : 36 time for calcul the mask position with numpy : 0.0012178421020507812 nb_pixel_total : 65723 time to create 1 rle with old method : 0.0718536376953125 length of segment : 432 time for calcul the mask position with numpy : 0.00015687942504882812 nb_pixel_total : 6695 time to create 1 rle with old method : 0.0077435970306396484 length of segment : 86 time for calcul the mask position with numpy : 0.00016021728515625 nb_pixel_total : 6647 time to create 1 rle with old method : 0.007654666900634766 length of segment : 130 time for calcul the mask position with numpy : 0.0001437664031982422 nb_pixel_total : 5237 time to create 1 rle with old method : 0.006398439407348633 length of segment : 84 time for calcul the mask position with numpy : 0.0008745193481445312 nb_pixel_total : 41764 time to create 1 rle with old method : 0.04579472541809082 length of segment : 252 time for calcul the mask position with numpy : 0.00029468536376953125 nb_pixel_total : 13854 time to create 1 rle with old method : 0.016542911529541016 length of segment : 124 time for calcul the mask position with numpy : 8.869171142578125e-05 nb_pixel_total : 2861 time to create 1 rle with old method : 0.0033762454986572266 length of segment : 69 time for calcul the mask position with numpy : 0.0001780986785888672 nb_pixel_total : 6443 time to create 1 rle with old method : 0.007577419281005859 length of segment : 106 time for calcul the mask position with numpy : 0.0007994174957275391 nb_pixel_total : 42482 time to create 1 rle with old method : 0.046949148178100586 length of segment : 221 time for calcul the mask position with numpy : 0.006367921829223633 nb_pixel_total : 388777 time to create 1 rle with new method : 0.024491548538208008 length of segment : 768 time for calcul the mask position with numpy : 0.0006849765777587891 nb_pixel_total : 35959 time to create 1 rle with old method : 0.039589881896972656 length of segment : 305 time for calcul the mask position with numpy : 0.0003330707550048828 nb_pixel_total : 15518 time to create 1 rle with old method : 0.01750779151916504 length of segment : 125 time for calcul the mask position with numpy : 0.00019598007202148438 nb_pixel_total : 8632 time to create 1 rle with old method : 0.009450197219848633 length of segment : 153 time for calcul the mask position with numpy : 0.0015609264373779297 nb_pixel_total : 66280 time to create 1 rle with old method : 0.06953048706054688 length of segment : 378 time for calcul the mask position with numpy : 0.0006308555603027344 nb_pixel_total : 35441 time to create 1 rle with old method : 0.03954792022705078 length of segment : 186 time for calcul the mask position with numpy : 0.0004153251647949219 nb_pixel_total : 24111 time to create 1 rle with old method : 0.02666950225830078 length of segment : 214 time for calcul the mask position with numpy : 0.004945516586303711 nb_pixel_total : 295535 time to create 1 rle with new method : 0.019987821578979492 length of segment : 909 time for calcul the mask position with numpy : 0.0003151893615722656 nb_pixel_total : 13194 time to create 1 rle with old method : 0.014857053756713867 length of segment : 200 time for calcul the mask position with numpy : 0.00013899803161621094 nb_pixel_total : 4728 time to create 1 rle with old method : 0.005532979965209961 length of segment : 75 time for calcul the mask position with numpy : 0.0002155303955078125 nb_pixel_total : 8945 time to create 1 rle with old method : 0.010111331939697266 length of segment : 130 time for calcul the mask position with numpy : 0.003137826919555664 nb_pixel_total : 275413 time to create 1 rle with new method : 0.012164115905761719 length of segment : 509 time for calcul the mask position with numpy : 0.0008158683776855469 nb_pixel_total : 33469 time to create 1 rle with old method : 0.03575873374938965 length of segment : 393 time for calcul the mask position with numpy : 0.00017070770263671875 nb_pixel_total : 4850 time to create 1 rle with old method : 0.005905628204345703 length of segment : 92 time for calcul the mask position with numpy : 0.0012612342834472656 nb_pixel_total : 78924 time to create 1 rle with old method : 0.08880162239074707 length of segment : 366 time for calcul the mask position with numpy : 0.00028395652770996094 nb_pixel_total : 15767 time to create 1 rle with old method : 0.01830148696899414 length of segment : 97 time for calcul the mask position with numpy : 0.0003952980041503906 nb_pixel_total : 20414 time to create 1 rle with old method : 0.022402048110961914 length of segment : 229 time for calcul the mask position with numpy : 0.001975536346435547 nb_pixel_total : 82336 time to create 1 rle with old method : 0.09462618827819824 length of segment : 618 time for calcul the mask position with numpy : 0.00021147727966308594 nb_pixel_total : 8018 time to create 1 rle with old method : 0.01018214225769043 length of segment : 83 time for calcul the mask position with numpy : 0.0018262863159179688 nb_pixel_total : 79100 time to create 1 rle with old method : 0.08858013153076172 length of segment : 426 time for calcul the mask position with numpy : 0.0019369125366210938 nb_pixel_total : 134279 time to create 1 rle with old method : 0.14819574356079102 length of segment : 374 time for calcul the mask position with numpy : 0.003051280975341797 nb_pixel_total : 185779 time to create 1 rle with new method : 0.013265848159790039 length of segment : 651 time for calcul the mask position with numpy : 0.0030727386474609375 nb_pixel_total : 167256 time to create 1 rle with new method : 0.012537717819213867 length of segment : 516 time for calcul the mask position with numpy : 0.0005652904510498047 nb_pixel_total : 21486 time to create 1 rle with old method : 0.02486896514892578 length of segment : 237 time for calcul the mask position with numpy : 0.00587916374206543 nb_pixel_total : 246725 time to create 1 rle with new method : 0.02381730079650879 length of segment : 717 time for calcul the mask position with numpy : 0.00139617919921875 nb_pixel_total : 67046 time to create 1 rle with old method : 0.07628154754638672 length of segment : 428 time for calcul the mask position with numpy : 0.002179861068725586 nb_pixel_total : 115665 time to create 1 rle with old method : 0.13504743576049805 length of segment : 435 time for calcul the mask position with numpy : 0.0006392002105712891 nb_pixel_total : 24133 time to create 1 rle with old method : 0.028541088104248047 length of segment : 202 time for calcul the mask position with numpy : 0.0003428459167480469 nb_pixel_total : 5322 time to create 1 rle with old method : 0.0061571598052978516 length of segment : 98 time for calcul the mask position with numpy : 0.00021147727966308594 nb_pixel_total : 7043 time to create 1 rle with old method : 0.008226871490478516 length of segment : 106 time for calcul the mask position with numpy : 0.00022602081298828125 nb_pixel_total : 8441 time to create 1 rle with old method : 0.009909868240356445 length of segment : 129 time for calcul the mask position with numpy : 0.00018858909606933594 nb_pixel_total : 9075 time to create 1 rle with old method : 0.010565757751464844 length of segment : 111 time for calcul the mask position with numpy : 0.0007379055023193359 nb_pixel_total : 35415 time to create 1 rle with old method : 0.041210174560546875 length of segment : 215 time for calcul the mask position with numpy : 0.0003814697265625 nb_pixel_total : 8085 time to create 1 rle with old method : 0.009619474411010742 length of segment : 111 time for calcul the mask position with numpy : 0.006425619125366211 nb_pixel_total : 406869 time to create 1 rle with new method : 0.02272653579711914 length of segment : 601 time for calcul the mask position with numpy : 0.0007574558258056641 nb_pixel_total : 30465 time to create 1 rle with old method : 0.034960031509399414 length of segment : 327 time for calcul the mask position with numpy : 0.0021452903747558594 nb_pixel_total : 125405 time to create 1 rle with old method : 0.13213586807250977 length of segment : 475 time for calcul the mask position with numpy : 0.0012485980987548828 nb_pixel_total : 52473 time to create 1 rle with old method : 0.059996843338012695 length of segment : 340 time for calcul the mask position with numpy : 0.001476287841796875 nb_pixel_total : 46795 time to create 1 rle with old method : 0.05493450164794922 length of segment : 282 time for calcul the mask position with numpy : 0.004593610763549805 nb_pixel_total : 176004 time to create 1 rle with new method : 0.011725664138793945 length of segment : 536 time for calcul the mask position with numpy : 0.005846500396728516 nb_pixel_total : 104630 time to create 1 rle with old method : 0.11371707916259766 length of segment : 488 time for calcul the mask position with numpy : 0.0006542205810546875 nb_pixel_total : 11890 time to create 1 rle with old method : 0.01361227035522461 length of segment : 199 time for calcul the mask position with numpy : 0.011358022689819336 nb_pixel_total : 406726 time to create 1 rle with new method : 0.022136449813842773 length of segment : 626 time for calcul the mask position with numpy : 0.002069234848022461 nb_pixel_total : 53609 time to create 1 rle with old method : 0.05735373497009277 length of segment : 299 time for calcul the mask position with numpy : 0.0005135536193847656 nb_pixel_total : 7328 time to create 1 rle with old method : 0.008685827255249023 length of segment : 112 time for calcul the mask position with numpy : 0.0026400089263916016 nb_pixel_total : 59015 time to create 1 rle with old method : 0.06656861305236816 length of segment : 375 time for calcul the mask position with numpy : 0.0004317760467529297 nb_pixel_total : 7485 time to create 1 rle with old method : 0.008890151977539062 length of segment : 80 time for calcul the mask position with numpy : 0.0007929801940917969 nb_pixel_total : 13925 time to create 1 rle with old method : 0.01634669303894043 length of segment : 130 time for calcul the mask position with numpy : 0.00035691261291503906 nb_pixel_total : 7113 time to create 1 rle with old method : 0.00833892822265625 length of segment : 73 time for calcul the mask position with numpy : 0.002009153366088867 nb_pixel_total : 54244 time to create 1 rle with old method : 0.059217214584350586 length of segment : 318 time for calcul the mask position with numpy : 0.0028793811798095703 nb_pixel_total : 54320 time to create 1 rle with old method : 0.059954166412353516 length of segment : 297 time for calcul the mask position with numpy : 0.0013766288757324219 nb_pixel_total : 34094 time to create 1 rle with old method : 0.03851437568664551 length of segment : 392 time for calcul the mask position with numpy : 0.0015926361083984375 nb_pixel_total : 38279 time to create 1 rle with old method : 0.042098283767700195 length of segment : 274 time for calcul the mask position with numpy : 0.010752677917480469 nb_pixel_total : 318276 time to create 1 rle with new method : 0.025315046310424805 length of segment : 1291 time for calcul the mask position with numpy : 0.0012123584747314453 nb_pixel_total : 38605 time to create 1 rle with old method : 0.041335105895996094 length of segment : 214 time for calcul the mask position with numpy : 0.0076677799224853516 nb_pixel_total : 158816 time to create 1 rle with new method : 0.01953577995300293 length of segment : 626 time for calcul the mask position with numpy : 0.0009379386901855469 nb_pixel_total : 20451 time to create 1 rle with old method : 0.02280116081237793 length of segment : 187 time for calcul the mask position with numpy : 0.005040168762207031 nb_pixel_total : 269708 time to create 1 rle with new method : 0.017089366912841797 length of segment : 540 time for calcul the mask position with numpy : 0.002289295196533203 nb_pixel_total : 67101 time to create 1 rle with old method : 0.06990575790405273 length of segment : 404 time for calcul the mask position with numpy : 0.0002067089080810547 nb_pixel_total : 5515 time to create 1 rle with old method : 0.006595134735107422 length of segment : 78 time for calcul the mask position with numpy : 0.00019097328186035156 nb_pixel_total : 2891 time to create 1 rle with old method : 0.003534078598022461 length of segment : 52 time for calcul the mask position with numpy : 0.002928018569946289 nb_pixel_total : 82560 time to create 1 rle with old method : 0.09239625930786133 length of segment : 457 time for calcul the mask position with numpy : 0.0003647804260253906 nb_pixel_total : 8969 time to create 1 rle with old method : 0.010599136352539062 length of segment : 90 time for calcul the mask position with numpy : 0.0002827644348144531 nb_pixel_total : 5291 time to create 1 rle with old method : 0.006362199783325195 length of segment : 88 time for calcul the mask position with numpy : 0.0003352165222167969 nb_pixel_total : 5697 time to create 1 rle with old method : 0.006838083267211914 length of segment : 164 time for calcul the mask position with numpy : 0.0004038810729980469 nb_pixel_total : 17730 time to create 1 rle with old method : 0.019896507263183594 length of segment : 151 time for calcul the mask position with numpy : 0.0022835731506347656 nb_pixel_total : 72253 time to create 1 rle with old method : 0.0786600112915039 length of segment : 569 time for calcul the mask position with numpy : 0.003091573715209961 nb_pixel_total : 66335 time to create 1 rle with old method : 0.07097363471984863 length of segment : 374 time for calcul the mask position with numpy : 0.0005908012390136719 nb_pixel_total : 18882 time to create 1 rle with old method : 0.02180171012878418 length of segment : 205 time for calcul the mask position with numpy : 0.0004341602325439453 nb_pixel_total : 20767 time to create 1 rle with old method : 0.024382591247558594 length of segment : 186 time for calcul the mask position with numpy : 0.0006339550018310547 nb_pixel_total : 24596 time to create 1 rle with old method : 0.027431488037109375 length of segment : 204 time for calcul the mask position with numpy : 0.0579073429107666 nb_pixel_total : 3132904 time to create 1 rle with new method : 0.11250495910644531 length of segment : 1997 time for calcul the mask position with numpy : 0.0022552013397216797 nb_pixel_total : 193917 time to create 1 rle with new method : 0.008601188659667969 length of segment : 643 time for calcul the mask position with numpy : 0.0012464523315429688 nb_pixel_total : 74534 time to create 1 rle with old method : 0.08078169822692871 length of segment : 385 time for calcul the mask position with numpy : 0.0024023056030273438 nb_pixel_total : 58591 time to create 1 rle with old method : 0.0699002742767334 length of segment : 283 time for calcul the mask position with numpy : 0.0006988048553466797 nb_pixel_total : 12883 time to create 1 rle with old method : 0.014844179153442383 length of segment : 217 time for calcul the mask position with numpy : 0.0007195472717285156 nb_pixel_total : 14696 time to create 1 rle with old method : 0.017332077026367188 length of segment : 249 time for calcul the mask position with numpy : 0.0003020763397216797 nb_pixel_total : 4846 time to create 1 rle with old method : 0.005870819091796875 length of segment : 84 time for calcul the mask position with numpy : 0.005396842956542969 nb_pixel_total : 167475 time to create 1 rle with new method : 0.00859379768371582 length of segment : 670 time for calcul the mask position with numpy : 0.00074005126953125 nb_pixel_total : 21749 time to create 1 rle with old method : 0.025273799896240234 length of segment : 161 time for calcul the mask position with numpy : 0.009130001068115234 nb_pixel_total : 257703 time to create 1 rle with new method : 0.015746116638183594 length of segment : 893 time for calcul the mask position with numpy : 0.012476205825805664 nb_pixel_total : 481845 time to create 1 rle with new method : 0.017752408981323242 length of segment : 956 time for calcul the mask position with numpy : 0.0012767314910888672 nb_pixel_total : 32593 time to create 1 rle with old method : 0.03775143623352051 length of segment : 335 time for calcul the mask position with numpy : 0.0020716190338134766 nb_pixel_total : 59354 time to create 1 rle with old method : 0.06684446334838867 length of segment : 537 time for calcul the mask position with numpy : 0.0001709461212158203 nb_pixel_total : 1293 time to create 1 rle with old method : 0.002321481704711914 length of segment : 37 time for calcul the mask position with numpy : 0.0009593963623046875 nb_pixel_total : 45554 time to create 1 rle with old method : 0.05973649024963379 length of segment : 277 time for calcul the mask position with numpy : 0.004285573959350586 nb_pixel_total : 130249 time to create 1 rle with old method : 0.14087390899658203 length of segment : 517 time for calcul the mask position with numpy : 0.0003299713134765625 nb_pixel_total : 3983 time to create 1 rle with old method : 0.0047299861907958984 length of segment : 155 time for calcul the mask position with numpy : 0.00013589859008789062 nb_pixel_total : 1940 time to create 1 rle with old method : 0.0025072097778320312 length of segment : 41 time for calcul the mask position with numpy : 0.0038022994995117188 nb_pixel_total : 186537 time to create 1 rle with new method : 0.00815129280090332 length of segment : 556 time for calcul the mask position with numpy : 0.000873565673828125 nb_pixel_total : 26354 time to create 1 rle with old method : 0.03074359893798828 length of segment : 240 time for calcul the mask position with numpy : 0.0009634494781494141 nb_pixel_total : 54896 time to create 1 rle with old method : 0.06255912780761719 length of segment : 278 time for calcul the mask position with numpy : 0.00034046173095703125 nb_pixel_total : 15661 time to create 1 rle with old method : 0.017881155014038086 length of segment : 159 time for calcul the mask position with numpy : 0.0009171962738037109 nb_pixel_total : 26935 time to create 1 rle with old method : 0.030622482299804688 length of segment : 167 time for calcul the mask position with numpy : 0.0061397552490234375 nb_pixel_total : 159029 time to create 1 rle with new method : 0.010046243667602539 length of segment : 651 time for calcul the mask position with numpy : 0.0044820308685302734 nb_pixel_total : 259328 time to create 1 rle with new method : 0.013689517974853516 length of segment : 960 time for calcul the mask position with numpy : 0.0026025772094726562 nb_pixel_total : 152888 time to create 1 rle with new method : 0.006432533264160156 length of segment : 608 time for calcul the mask position with numpy : 0.0005767345428466797 nb_pixel_total : 26875 time to create 1 rle with old method : 0.030057907104492188 length of segment : 171 time for calcul the mask position with numpy : 0.0001842975616455078 nb_pixel_total : 2390 time to create 1 rle with old method : 0.002816915512084961 length of segment : 63 time for calcul the mask position with numpy : 0.01940321922302246 nb_pixel_total : 644022 time to create 1 rle with new method : 0.032695770263671875 length of segment : 1369 time for calcul the mask position with numpy : 0.0023665428161621094 nb_pixel_total : 82944 time to create 1 rle with old method : 0.09763360023498535 length of segment : 488 time for calcul the mask position with numpy : 0.000270843505859375 nb_pixel_total : 6648 time to create 1 rle with old method : 0.00816798210144043 length of segment : 76 time for calcul the mask position with numpy : 0.0014045238494873047 nb_pixel_total : 36288 time to create 1 rle with old method : 0.04391169548034668 length of segment : 158 time for calcul the mask position with numpy : 0.0006284713745117188 nb_pixel_total : 6749 time to create 1 rle with old method : 0.008771657943725586 length of segment : 132 time for calcul the mask position with numpy : 0.0024025440216064453 nb_pixel_total : 66015 time to create 1 rle with old method : 0.07601118087768555 length of segment : 352 time for calcul the mask position with numpy : 0.0004634857177734375 nb_pixel_total : 15288 time to create 1 rle with old method : 0.018332481384277344 length of segment : 134 time for calcul the mask position with numpy : 0.0030324459075927734 nb_pixel_total : 99099 time to create 1 rle with old method : 0.12324237823486328 length of segment : 385 time for calcul the mask position with numpy : 0.002758026123046875 nb_pixel_total : 91488 time to create 1 rle with old method : 0.10738086700439453 length of segment : 290 time for calcul the mask position with numpy : 0.002665281295776367 nb_pixel_total : 72060 time to create 1 rle with old method : 0.0807034969329834 length of segment : 567 time for calcul the mask position with numpy : 0.00037860870361328125 nb_pixel_total : 6516 time to create 1 rle with old method : 0.007471323013305664 length of segment : 141 time for calcul the mask position with numpy : 0.0003342628479003906 nb_pixel_total : 8027 time to create 1 rle with old method : 0.01024627685546875 length of segment : 104 time for calcul the mask position with numpy : 0.0024437904357910156 nb_pixel_total : 51369 time to create 1 rle with old method : 0.05999946594238281 length of segment : 436 time for calcul the mask position with numpy : 0.00016689300537109375 nb_pixel_total : 1625 time to create 1 rle with old method : 0.0020842552185058594 length of segment : 52 time for calcul the mask position with numpy : 0.0025854110717773438 nb_pixel_total : 90508 time to create 1 rle with old method : 0.09932518005371094 length of segment : 474 time for calcul the mask position with numpy : 0.006648540496826172 nb_pixel_total : 236769 time to create 1 rle with new method : 0.015661239624023438 length of segment : 828 time for calcul the mask position with numpy : 0.0025739669799804688 nb_pixel_total : 108147 time to create 1 rle with old method : 0.12044906616210938 length of segment : 370 time for calcul the mask position with numpy : 0.0006134510040283203 nb_pixel_total : 7292 time to create 1 rle with old method : 0.008990287780761719 length of segment : 199 time for calcul the mask position with numpy : 0.0010180473327636719 nb_pixel_total : 25879 time to create 1 rle with old method : 0.040329933166503906 length of segment : 205 time for calcul the mask position with numpy : 0.00020432472229003906 nb_pixel_total : 1572 time to create 1 rle with old method : 0.0028493404388427734 length of segment : 47 time for calcul the mask position with numpy : 0.00014162063598632812 nb_pixel_total : 2893 time to create 1 rle with old method : 0.003980398178100586 length of segment : 60 time for calcul the mask position with numpy : 0.003782510757446289 nb_pixel_total : 139825 time to create 1 rle with old method : 0.15871596336364746 length of segment : 516 time for calcul the mask position with numpy : 0.0001804828643798828 nb_pixel_total : 1468 time to create 1 rle with old method : 0.0021390914916992188 length of segment : 67 time for calcul the mask position with numpy : 0.00037360191345214844 nb_pixel_total : 7401 time to create 1 rle with old method : 0.009888887405395508 length of segment : 143 time for calcul the mask position with numpy : 0.02014946937561035 nb_pixel_total : 554364 time to create 1 rle with new method : 0.045923709869384766 length of segment : 1701 time for calcul the mask position with numpy : 0.0010237693786621094 nb_pixel_total : 65819 time to create 1 rle with old method : 0.07877516746520996 length of segment : 316 time for calcul the mask position with numpy : 0.001893758773803711 nb_pixel_total : 114866 time to create 1 rle with old method : 0.13435912132263184 length of segment : 610 time for calcul the mask position with numpy : 0.003347158432006836 nb_pixel_total : 154233 time to create 1 rle with new method : 0.009834527969360352 length of segment : 811 time for calcul the mask position with numpy : 0.00046944618225097656 nb_pixel_total : 18847 time to create 1 rle with old method : 0.030600786209106445 length of segment : 171 time for calcul the mask position with numpy : 0.0003390312194824219 nb_pixel_total : 23903 time to create 1 rle with old method : 0.02717113494873047 length of segment : 172 time for calcul the mask position with numpy : 0.0110015869140625 nb_pixel_total : 680061 time to create 1 rle with new method : 0.0349736213684082 length of segment : 1467 time for calcul the mask position with numpy : 0.05993962287902832 nb_pixel_total : 4195806 time to create 1 rle with new method : 0.1131296157836914 length of segment : 2865 time for calcul the mask position with numpy : 0.00010728836059570312 nb_pixel_total : 2476 time to create 1 rle with old method : 0.0037841796875 length of segment : 65 time for calcul the mask position with numpy : 0.0030364990234375 nb_pixel_total : 226225 time to create 1 rle with new method : 0.007520198822021484 length of segment : 555 time for calcul the mask position with numpy : 0.0008666515350341797 nb_pixel_total : 45395 time to create 1 rle with old method : 0.05271601676940918 length of segment : 256 time for calcul the mask position with numpy : 0.00016164779663085938 nb_pixel_total : 1767 time to create 1 rle with old method : 0.003366231918334961 length of segment : 40 time spent for convertir_results : 15.241495847702026 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 174 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 55749 save missing photos in datou_result : time spend for datou_step_exec : 53.93087410926819 time spend to save output : 9.355742454528809 total time spend for step 1 : 63.286616563797 step2:crop_condition Thu Feb 6 02:17:34 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 174 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 13 About to insert : list_path_to_insert length 13 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 12 About to insert : list_path_to_insert length 12 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 10 About to insert : list_path_to_insert length 10 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 40 About to insert : list_path_to_insert length 40 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1335035145, 1335035141, 1335035136, 1335035120, 1335007135, 1335007108, 1335006903, 1335006844, 1335006842, 1335006840] Looping around the photos to save general results len do output : 91 /-3656773804Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773805Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773819Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773815Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773820Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773828Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773830Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773837Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773855Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773854Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773884Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773878Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773913Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773791Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773786Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773808Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773810Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773817Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773826Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773856Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773850Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773857Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773874Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773876Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773928Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773793Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656908403Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773846Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773841Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773870Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773872Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773849Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656908413Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656908412Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773918Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773787Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773796Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773795Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656908402Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773809Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773799Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656908401Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773824Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773818Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773823Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773825Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773842Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773838Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773847Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773829Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656908405Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773843Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773858Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773859Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773871Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773862Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773861Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773865Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773885Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773894Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656908408Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773889Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656908410Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656908414Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656908411Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656908409Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773883Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773887Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773914Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773907Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773916Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773929Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773931Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773924Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773938Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773840Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773831Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773835Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773833Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773895Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773904Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773909Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773869Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773873Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773899Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773910Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773932Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773921Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773827Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773860Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3656773900Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035145', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035141', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035136', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035120', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335007135', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335007108', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006903', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006844', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006842', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006840', None, None, None, None, None, '2557221') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 283 time used for this insertion : 0.03567862510681152 save_final save missing photos in datou_result : time spend for datou_step_exec : 45.31171703338623 time spend to save output : 0.03863191604614258 total time spend for step 2 : 45.35034894943237 step3:thcl Thu Feb 6 02:18:20 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 2 we are using the classfication for multi_thcl [2456, 2868] time to import caffe and check if the image exist : 0.0050389766693115234 time to convert the images to numpy array : 0.02705216407775879 time to import caffe and check if the image exist : 0.013263702392578125 time to convert the images to numpy array : 0.24998807907104492 time to import caffe and check if the image exist : 0.013092756271362305 time to convert the images to numpy array : 0.2635664939880371 time to import caffe and check if the image exist : 0.015362739562988281 time to convert the images to numpy array : 0.2843616008758545 time to import caffe and check if the image exist : 0.011350631713867188 time to convert the images to numpy array : 0.6134061813354492 time to import caffe and check if the image exist : 0.01351618766784668 time to convert the images to numpy array : 0.7240467071533203 time to import caffe and check if the image exist : 0.011993885040283203 time to convert the images to numpy array : 0.7445018291473389 time to import caffe and check if the image exist : 0.0076787471771240234 time to convert the images to numpy array : 0.7530162334442139 time to import caffe and check if the image exist : 0.007803440093994141 time to convert the images to numpy array : 0.7561924457550049 time to import caffe and check if the image exist : 0.014856815338134766 time to convert the images to numpy array : 0.8247106075286865 total time to convert the images to numpy array : 1.108625888824463 list photo_ids error: [] list photo_ids correct : [-3656773900, -3656773916, -3656773929, -3656773931, -3656773924, -3656773938, -3656773840, -3656773831, -3656773835, -3656773833, -3656773895, -3656773904, -3656773909, -3656773869, -3656773873, -3656773899, -3656773910, -3656773932, -3656773921, -3656773827, -3656773860, -3656908408, -3656773889, -3656908410, -3656908414, -3656908411, -3656908409, -3656773883, -3656773887, -3656773914, -3656773907, -3656773872, -3656773849, -3656908413, -3656908412, -3656773918, -3656773787, -3656773796, -3656773795, -3656908402, -3656773809, -3656773884, -3656773878, -3656773913, -3656773791, -3656773786, -3656773808, -3656773810, -3656773817, -3656773826, -3656773856, -3656773850, -3656773857, -3656773874, -3656773876, -3656773928, -3656773793, -3656908403, -3656773846, -3656773841, -3656773870, -3656773799, -3656908401, -3656773824, -3656773818, -3656773823, -3656773825, -3656773842, -3656773838, -3656773847, -3656773829, -3656773804, -3656773805, -3656773819, -3656773815, -3656773820, -3656773828, -3656773830, -3656773837, -3656773855, -3656773854, -3656908405, -3656773843, -3656773858, -3656773859, -3656773871, -3656773862, -3656773861, -3656773865, -3656773885, -3656773894] number of photos to traite : 91 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10774 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10555 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.4301066398620605 time used to do the prediction : 0.20815396308898926 we don't save the descriptors for this thcl 2456 tagging for thcl : 2868 To do loadFromThcl(), then load ParamDescType : thcl2868 thcls : [{'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5288 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) To loadFromThcl() : net_5288 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10553 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_papier_nantes_300421 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_papier_nantes_300421 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_papier_nantes_300421 /data/models_weight/learn_papier_nantes_300421/caffemodel size_local : 44791983 size in s3 : 44791983 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:13 caffemodel already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/deploy.prototxt size_local : 17255 size in s3 : 17255 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:12 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:14 mean.npy already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/synset_words.txt size_local : 331 size in s3 : 331 create time local : 2021-08-09 05:56:00 create time in s3 : 2021-08-06 19:22:12 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_papier_nantes_300421/deploy.prototxt caffemodel_filename : /data/models_weight/learn_papier_nantes_300421/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10553 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.2420275211334229 time used to do the prediction : 0.17972445487976074 we don't save the descriptors for this thcl 2868 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 1.239776611328125e-05 save missing photos in datou_result : time spend for datou_step_exec : 11.069288492202759 time spend to save output : 0.0006785392761230469 total time spend for step 3 : 11.069967031478882 step4:merge_mask_thcl_custom Thu Feb 6 02:18:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 174 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1335035145, 1335035141, 1335035136, 1335035120, 1335007135, 1335007108, 1335006903, 1335006844, 1335006842, 1335006840] Looping around the photos to save general results len do output : 10 /1335035145Didn't retrieve data .Didn't retrieve data . /1335035141Didn't retrieve data .Didn't retrieve data . /1335035136Didn't retrieve data .Didn't retrieve data . /1335035120Didn't retrieve data .Didn't retrieve data . /1335007135Didn't retrieve data .Didn't retrieve data . /1335007108Didn't retrieve data .Didn't retrieve data . /1335006903Didn't retrieve data .Didn't retrieve data . /1335006844Didn't retrieve data .Didn't retrieve data . /1335006842Didn't retrieve data .Didn't retrieve data . /1335006840Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035145', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035141', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035136', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035120', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335007135', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335007108', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006903', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006844', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006842', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006840', None, None, None, None, None, '2557221') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.09462094306945801 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.09437417984008789 time spend to save output : 0.09512758255004883 total time spend for step 4 : 0.18950176239013672 step5:rle_unique_nms_with_priority Thu Feb 6 02:18:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 16 nb_hashtags : 4 time to prepare the origin masks : 7.880098104476929 time for calcul the mask position with numpy : 0.889641523361206 nb_pixel_total : 6485929 time to create 1 rle with new method : 1.49222731590271 time for calcul the mask position with numpy : 0.025133609771728516 nb_pixel_total : 5205 time to create 1 rle with old method : 0.006725311279296875 time for calcul the mask position with numpy : 0.024920225143432617 nb_pixel_total : 1012 time to create 1 rle with old method : 0.0012598037719726562 time for calcul the mask position with numpy : 0.024923324584960938 nb_pixel_total : 27 time to create 1 rle with old method : 0.00022029876708984375 time for calcul the mask position with numpy : 0.031565189361572266 nb_pixel_total : 2668 time to create 1 rle with old method : 0.004444599151611328 time for calcul the mask position with numpy : 0.02504587173461914 nb_pixel_total : 1460 time to create 1 rle with old method : 0.002427816390991211 time for calcul the mask position with numpy : 0.024468421936035156 nb_pixel_total : 6671 time to create 1 rle with old method : 0.008491754531860352 time for calcul the mask position with numpy : 0.02472209930419922 nb_pixel_total : 287744 time to create 1 rle with new method : 0.7116785049438477 time for calcul the mask position with numpy : 0.022902488708496094 nb_pixel_total : 14217 time to create 1 rle with old method : 0.016904830932617188 time for calcul the mask position with numpy : 0.02200007438659668 nb_pixel_total : 3628 time to create 1 rle with old method : 0.004546642303466797 time for calcul the mask position with numpy : 0.02716064453125 nb_pixel_total : 13185 time to create 1 rle with old method : 0.018876075744628906 time for calcul the mask position with numpy : 0.023887157440185547 nb_pixel_total : 65617 time to create 1 rle with old method : 0.08147001266479492 time for calcul the mask position with numpy : 0.026076078414916992 nb_pixel_total : 6110 time to create 1 rle with old method : 0.00712132453918457 time for calcul the mask position with numpy : 0.02355027198791504 nb_pixel_total : 35117 time to create 1 rle with old method : 0.03899335861206055 time for calcul the mask position with numpy : 0.02225518226623535 nb_pixel_total : 9329 time to create 1 rle with old method : 0.014919519424438477 time for calcul the mask position with numpy : 0.02379751205444336 nb_pixel_total : 1778 time to create 1 rle with old method : 0.002871990203857422 time for calcul the mask position with numpy : 0.025098323822021484 nb_pixel_total : 110543 time to create 1 rle with old method : 0.12294268608093262 create new chi : 3.8867061138153076 time to delete rle : 0.00984501838684082 batch 1 Loaded 17 chid ids of type : 3726 Number RLEs to save : 8308 TO DO : save crop sub photo not yet done ! save time : 1.0850608348846436 nb_obj : 14 nb_hashtags : 5 time to prepare the origin masks : 7.5881028175354 time for calcul the mask position with numpy : 0.43062305450439453 nb_pixel_total : 6059318 time to create 1 rle with new method : 1.0694756507873535 time for calcul the mask position with numpy : 0.02749180793762207 nb_pixel_total : 68448 time to create 1 rle with old method : 0.07528090476989746 time for calcul the mask position with numpy : 0.021670103073120117 nb_pixel_total : 24023 time to create 1 rle with old method : 0.029768705368041992 time for calcul the mask position with numpy : 0.02275228500366211 nb_pixel_total : 12829 time to create 1 rle with old method : 0.014118671417236328 time for calcul the mask position with numpy : 0.02213001251220703 nb_pixel_total : 8603 time to create 1 rle with old method : 0.011038780212402344 time for calcul the mask position with numpy : 0.022162914276123047 nb_pixel_total : 35302 time to create 1 rle with old method : 0.039574384689331055 time for calcul the mask position with numpy : 0.021983623504638672 nb_pixel_total : 15370 time to create 1 rle with old method : 0.017490625381469727 time for calcul the mask position with numpy : 0.020738601684570312 nb_pixel_total : 35885 time to create 1 rle with old method : 0.041115522384643555 time for calcul the mask position with numpy : 0.024866819381713867 nb_pixel_total : 388305 time to create 1 rle with new method : 0.7322471141815186 time for calcul the mask position with numpy : 0.025316953659057617 nb_pixel_total : 6427 time to create 1 rle with old method : 0.007218122482299805 time for calcul the mask position with numpy : 0.022184133529663086 nb_pixel_total : 2837 time to create 1 rle with old method : 0.0038971900939941406 time for calcul the mask position with numpy : 0.022718191146850586 nb_pixel_total : 41658 time to create 1 rle with old method : 0.0493314266204834 time for calcul the mask position with numpy : 0.02346658706665039 nb_pixel_total : 42316 time to create 1 rle with old method : 0.04768657684326172 time for calcul the mask position with numpy : 0.023319721221923828 nb_pixel_total : 13710 time to create 1 rle with old method : 0.015192985534667969 time for calcul the mask position with numpy : 0.025677919387817383 nb_pixel_total : 295209 time to create 1 rle with new method : 0.6642496585845947 create new chi : 3.6733593940734863 time to delete rle : 0.0009350776672363281 batch 1 Loaded 15 chid ids of type : 3726 Number RLEs to save : 9943 TO DO : save crop sub photo not yet done ! save time : 0.8816990852355957 nb_obj : 15 nb_hashtags : 5 time to prepare the origin masks : 8.755747318267822 time for calcul the mask position with numpy : 0.3037285804748535 nb_pixel_total : 6196711 time to create 1 rle with new method : 0.7318007946014404 time for calcul the mask position with numpy : 0.024125099182128906 nb_pixel_total : 125 time to create 1 rle with old method : 0.0003333091735839844 time for calcul the mask position with numpy : 0.02426004409790039 nb_pixel_total : 21401 time to create 1 rle with old method : 0.026066303253173828 time for calcul the mask position with numpy : 0.025052309036254883 nb_pixel_total : 128397 time to create 1 rle with old method : 0.14919090270996094 time for calcul the mask position with numpy : 0.02193284034729004 nb_pixel_total : 7905 time to create 1 rle with old method : 0.009274721145629883 time for calcul the mask position with numpy : 0.022605419158935547 nb_pixel_total : 10434 time to create 1 rle with old method : 0.011989355087280273 time for calcul the mask position with numpy : 0.024448633193969727 nb_pixel_total : 15714 time to create 1 rle with old method : 0.018040895462036133 time for calcul the mask position with numpy : 0.02668905258178711 nb_pixel_total : 78774 time to create 1 rle with old method : 0.08869242668151855 time for calcul the mask position with numpy : 0.02608799934387207 nb_pixel_total : 33000 time to create 1 rle with old method : 0.04043459892272949 time for calcul the mask position with numpy : 0.028325557708740234 nb_pixel_total : 78955 time to create 1 rle with old method : 0.09000945091247559 time for calcul the mask position with numpy : 0.025782108306884766 nb_pixel_total : 274981 time to create 1 rle with new method : 0.6262893676757812 time for calcul the mask position with numpy : 0.025064706802368164 nb_pixel_total : 4788 time to create 1 rle with old method : 0.006074190139770508 time for calcul the mask position with numpy : 0.023282527923583984 nb_pixel_total : 201 time to create 1 rle with old method : 0.0006358623504638672 time for calcul the mask position with numpy : 0.02412581443786621 nb_pixel_total : 185224 time to create 1 rle with new method : 0.540522575378418 time for calcul the mask position with numpy : 0.02960038185119629 nb_pixel_total : 8918 time to create 1 rle with old method : 0.010401248931884766 time for calcul the mask position with numpy : 0.03178715705871582 nb_pixel_total : 4712 time to create 1 rle with old method : 0.00554656982421875 create new chi : 3.1442408561706543 time to delete rle : 0.0012569427490234375 batch 1 Loaded 16 chid ids of type : 3726 Number RLEs to save : 9961 TO DO : save crop sub photo not yet done ! save time : 0.6298191547393799 nb_obj : 20 nb_hashtags : 5 time to prepare the origin masks : 11.4395911693573 time for calcul the mask position with numpy : 0.2750422954559326 nb_pixel_total : 5582795 time to create 1 rle with new method : 0.15545320510864258 time for calcul the mask position with numpy : 0.02162027359008789 nb_pixel_total : 115105 time to create 1 rle with old method : 0.1203460693359375 time for calcul the mask position with numpy : 0.020984649658203125 nb_pixel_total : 1685 time to create 1 rle with old method : 0.0020203590393066406 time for calcul the mask position with numpy : 0.02168560028076172 nb_pixel_total : 23834 time to create 1 rle with old method : 0.02572488784790039 time for calcul the mask position with numpy : 0.03412461280822754 nb_pixel_total : 66998 time to create 1 rle with old method : 0.07054781913757324 time for calcul the mask position with numpy : 0.03351330757141113 nb_pixel_total : 231 time to create 1 rle with old method : 0.0004253387451171875 time for calcul the mask position with numpy : 0.03310394287109375 nb_pixel_total : 30509 time to create 1 rle with old method : 0.03180670738220215 time for calcul the mask position with numpy : 0.03404545783996582 nb_pixel_total : 35129 time to create 1 rle with old method : 0.0381617546081543 time for calcul the mask position with numpy : 0.03448915481567383 nb_pixel_total : 7006 time to create 1 rle with old method : 0.007729053497314453 time for calcul the mask position with numpy : 0.03225898742675781 nb_pixel_total : 8045 time to create 1 rle with old method : 0.009024858474731445 time for calcul the mask position with numpy : 0.03563714027404785 nb_pixel_total : 124936 time to create 1 rle with old method : 0.13497233390808105 time for calcul the mask position with numpy : 0.03211069107055664 nb_pixel_total : 6107 time to create 1 rle with old method : 0.007589817047119141 time for calcul the mask position with numpy : 0.033065080642700195 nb_pixel_total : 8374 time to create 1 rle with old method : 0.009115219116210938 time for calcul the mask position with numpy : 0.03281593322753906 nb_pixel_total : 5290 time to create 1 rle with old method : 0.006112575531005859 time for calcul the mask position with numpy : 0.03538155555725098 nb_pixel_total : 245877 time to create 1 rle with new method : 0.13411545753479004 time for calcul the mask position with numpy : 0.03119945526123047 nb_pixel_total : 405666 time to create 1 rle with new method : 0.1331193447113037 time for calcul the mask position with numpy : 0.022013425827026367 nb_pixel_total : 2630 time to create 1 rle with old method : 0.003367900848388672 time for calcul the mask position with numpy : 0.020384550094604492 nb_pixel_total : 46634 time to create 1 rle with old method : 0.050942420959472656 time for calcul the mask position with numpy : 0.021983623504638672 nb_pixel_total : 175540 time to create 1 rle with new method : 0.1446399688720703 time for calcul the mask position with numpy : 0.02310013771057129 nb_pixel_total : 53452 time to create 1 rle with old method : 0.05625319480895996 time for calcul the mask position with numpy : 0.021167993545532227 nb_pixel_total : 104397 time to create 1 rle with old method : 0.10978293418884277 create new chi : 2.1747875213623047 time to delete rle : 0.001581430435180664 batch 1 Loaded 21 chid ids of type : 3726 Number RLEs to save : 14614 TO DO : save crop sub photo not yet done ! save time : 1.5195884704589844 nb_obj : 26 nb_hashtags : 7 time to prepare the origin masks : 9.978515148162842 time for calcul the mask position with numpy : 0.8238856792449951 nb_pixel_total : 5892414 time to create 1 rle with new method : 0.9624462127685547 time for calcul the mask position with numpy : 0.03171658515930176 nb_pixel_total : 2872 time to create 1 rle with old method : 0.003981113433837891 time for calcul the mask position with numpy : 0.03726840019226074 nb_pixel_total : 34063 time to create 1 rle with old method : 0.056165456771850586 time for calcul the mask position with numpy : 0.03418588638305664 nb_pixel_total : 13831 time to create 1 rle with old method : 0.017494916915893555 time for calcul the mask position with numpy : 0.03890109062194824 nb_pixel_total : 58819 time to create 1 rle with old method : 0.07474088668823242 time for calcul the mask position with numpy : 0.033471107482910156 nb_pixel_total : 392 time to create 1 rle with old method : 0.0006465911865234375 time for calcul the mask position with numpy : 0.028911352157592773 nb_pixel_total : 20702 time to create 1 rle with old method : 0.023649930953979492 time for calcul the mask position with numpy : 0.02934551239013672 nb_pixel_total : 38482 time to create 1 rle with old method : 0.04280281066894531 time for calcul the mask position with numpy : 0.03015303611755371 nb_pixel_total : 82477 time to create 1 rle with old method : 0.11757946014404297 time for calcul the mask position with numpy : 0.041254520416259766 nb_pixel_total : 5231 time to create 1 rle with old method : 0.011344432830810547 time for calcul the mask position with numpy : 0.045398712158203125 nb_pixel_total : 18857 time to create 1 rle with old method : 0.03223061561584473 time for calcul the mask position with numpy : 0.03970074653625488 nb_pixel_total : 158500 time to create 1 rle with new method : 1.0569462776184082 time for calcul the mask position with numpy : 0.03900957107543945 nb_pixel_total : 7422 time to create 1 rle with old method : 0.008118867874145508 time for calcul the mask position with numpy : 0.0297393798828125 nb_pixel_total : 66204 time to create 1 rle with old method : 0.0737771987915039 time for calcul the mask position with numpy : 0.02912735939025879 nb_pixel_total : 7293 time to create 1 rle with old method : 0.00847005844116211 time for calcul the mask position with numpy : 0.030150175094604492 nb_pixel_total : 54090 time to create 1 rle with old method : 0.06979632377624512 time for calcul the mask position with numpy : 0.03909707069396973 nb_pixel_total : 17664 time to create 1 rle with old method : 0.027801513671875 time for calcul the mask position with numpy : 0.04327893257141113 nb_pixel_total : 100306 time to create 1 rle with old method : 0.11130738258361816 time for calcul the mask position with numpy : 0.02883458137512207 nb_pixel_total : 7075 time to create 1 rle with old method : 0.008175849914550781 time for calcul the mask position with numpy : 0.028535842895507812 nb_pixel_total : 38166 time to create 1 rle with old method : 0.04135274887084961 time for calcul the mask position with numpy : 0.03294944763183594 nb_pixel_total : 269337 time to create 1 rle with new method : 0.9053490161895752 time for calcul the mask position with numpy : 0.027988433837890625 nb_pixel_total : 5427 time to create 1 rle with old method : 0.006158351898193359 time for calcul the mask position with numpy : 0.028260469436645508 nb_pixel_total : 74593 time to create 1 rle with old method : 0.08094000816345215 time for calcul the mask position with numpy : 0.02943110466003418 nb_pixel_total : 66921 time to create 1 rle with old method : 0.07472896575927734 time for calcul the mask position with numpy : 0.02882695198059082 nb_pixel_total : 83 time to create 1 rle with old method : 0.0003712177276611328 time for calcul the mask position with numpy : 0.03164172172546387 nb_pixel_total : 8971 time to create 1 rle with old method : 0.010997772216796875 time for calcul the mask position with numpy : 0.029819965362548828 nb_pixel_total : 48 time to create 1 rle with old method : 0.0003077983856201172 create new chi : 5.617029190063477 time to delete rle : 0.0021533966064453125 batch 1 Loaded 27 chid ids of type : 3726 Number RLEs to save : 15540 TO DO : save crop sub photo not yet done ! save time : 4.506143808364868 nb_obj : 4 nb_hashtags : 3 time to prepare the origin masks : 0.9774794578552246 time for calcul the mask position with numpy : 0.06258797645568848 nb_pixel_total : 3641534 time to create 1 rle with new method : 0.4858818054199219 time for calcul the mask position with numpy : 0.022602319717407227 nb_pixel_total : 74221 time to create 1 rle with old method : 0.07535314559936523 time for calcul the mask position with numpy : 0.023205041885375977 nb_pixel_total : 193730 time to create 1 rle with new method : 0.1958022117614746 time for calcul the mask position with numpy : 0.022962570190429688 nb_pixel_total : 11774 time to create 1 rle with old method : 0.012393951416015625 time for calcul the mask position with numpy : 0.060939788818359375 nb_pixel_total : 3128981 time to create 1 rle with new method : 0.3131563663482666 create new chi : 1.349616289138794 time to delete rle : 0.001214742660522461 batch 1 Loaded 5 chid ids of type : 3726 Number RLEs to save : 8398 TO DO : save crop sub photo not yet done ! save time : 1.330965280532837 nb_obj : 24 nb_hashtags : 6 time to prepare the origin masks : 9.973294019699097 time for calcul the mask position with numpy : 0.20650267601013184 nb_pixel_total : 5328878 time to create 1 rle with new method : 0.5588235855102539 time for calcul the mask position with numpy : 0.02115035057067871 nb_pixel_total : 14435 time to create 1 rle with old method : 0.016675472259521484 time for calcul the mask position with numpy : 0.02549004554748535 nb_pixel_total : 480580 time to create 1 rle with new method : 0.13240408897399902 time for calcul the mask position with numpy : 0.020858049392700195 nb_pixel_total : 453 time to create 1 rle with old method : 0.0007221698760986328 time for calcul the mask position with numpy : 0.020748138427734375 nb_pixel_total : 15628 time to create 1 rle with old method : 0.017757892608642578 time for calcul the mask position with numpy : 0.02121138572692871 nb_pixel_total : 42098 time to create 1 rle with old method : 0.04692387580871582 time for calcul the mask position with numpy : 0.020953893661499023 nb_pixel_total : 21601 time to create 1 rle with old method : 0.024537086486816406 time for calcul the mask position with numpy : 0.020473003387451172 nb_pixel_total : 26311 time to create 1 rle with old method : 0.03277015686035156 time for calcul the mask position with numpy : 0.021356821060180664 nb_pixel_total : 129932 time to create 1 rle with old method : 0.1405320167541504 time for calcul the mask position with numpy : 0.020879030227661133 nb_pixel_total : 26781 time to create 1 rle with old method : 0.03081822395324707 time for calcul the mask position with numpy : 0.02311539649963379 nb_pixel_total : 250496 time to create 1 rle with new method : 0.14091849327087402 time for calcul the mask position with numpy : 0.029946088790893555 nb_pixel_total : 116224 time to create 1 rle with old method : 0.12980389595031738 time for calcul the mask position with numpy : 0.02223992347717285 nb_pixel_total : 1928 time to create 1 rle with old method : 0.00237274169921875 time for calcul the mask position with numpy : 0.023819446563720703 nb_pixel_total : 7163 time to create 1 rle with old method : 0.008774280548095703 time for calcul the mask position with numpy : 0.02353215217590332 nb_pixel_total : 186423 time to create 1 rle with new method : 0.8479971885681152 time for calcul the mask position with numpy : 0.034685373306274414 nb_pixel_total : 3973 time to create 1 rle with old method : 0.004578828811645508 time for calcul the mask position with numpy : 0.034763336181640625 nb_pixel_total : 54917 time to create 1 rle with old method : 0.06638073921203613 time for calcul the mask position with numpy : 0.03495907783508301 nb_pixel_total : 1283 time to create 1 rle with old method : 0.001528024673461914 time for calcul the mask position with numpy : 0.034922122955322266 nb_pixel_total : 159048 time to create 1 rle with new method : 0.13199615478515625 time for calcul the mask position with numpy : 0.037039756774902344 nb_pixel_total : 32541 time to create 1 rle with old method : 0.046399593353271484 time for calcul the mask position with numpy : 0.09316372871398926 nb_pixel_total : 4823 time to create 1 rle with old method : 0.008989334106445312 time for calcul the mask position with numpy : 0.05776190757751465 nb_pixel_total : 14528 time to create 1 rle with old method : 0.026844024658203125 time for calcul the mask position with numpy : 0.060591697692871094 nb_pixel_total : 58497 time to create 1 rle with old method : 0.07733941078186035 time for calcul the mask position with numpy : 0.030191898345947266 nb_pixel_total : 12781 time to create 1 rle with old method : 0.014334440231323242 time for calcul the mask position with numpy : 0.0232388973236084 nb_pixel_total : 58918 time to create 1 rle with old method : 0.06409382820129395 create new chi : 3.624551296234131 time to delete rle : 0.0018885135650634766 batch 1 Loaded 25 chid ids of type : 3726 Number RLEs to save : 18866 TO DO : save crop sub photo not yet done ! save time : 1.5209379196166992 nb_obj : 19 nb_hashtags : 5 time to prepare the origin masks : 8.741664171218872 time for calcul the mask position with numpy : 0.39229416847229004 nb_pixel_total : 5173841 time to create 1 rle with new method : 1.106978178024292 time for calcul the mask position with numpy : 0.025441408157348633 nb_pixel_total : 90315 time to create 1 rle with old method : 0.0963447093963623 time for calcul the mask position with numpy : 0.0226438045501709 nb_pixel_total : 1239 time to create 1 rle with old method : 0.001878976821899414 time for calcul the mask position with numpy : 0.029335975646972656 nb_pixel_total : 425522 time to create 1 rle with new method : 0.5054957866668701 time for calcul the mask position with numpy : 0.032053470611572266 nb_pixel_total : 7986 time to create 1 rle with old method : 0.008197784423828125 time for calcul the mask position with numpy : 0.0227811336517334 nb_pixel_total : 6624 time to create 1 rle with old method : 0.006911754608154297 time for calcul the mask position with numpy : 0.024704933166503906 nb_pixel_total : 162004 time to create 1 rle with new method : 0.48268675804138184 time for calcul the mask position with numpy : 0.025315046310424805 nb_pixel_total : 1622 time to create 1 rle with old method : 0.001787424087524414 time for calcul the mask position with numpy : 0.02315068244934082 nb_pixel_total : 108037 time to create 1 rle with old method : 0.10783004760742188 time for calcul the mask position with numpy : 0.03588294982910156 nb_pixel_total : 6503 time to create 1 rle with old method : 0.0068073272705078125 time for calcul the mask position with numpy : 0.034671783447265625 nb_pixel_total : 71927 time to create 1 rle with old method : 0.07272696495056152 time for calcul the mask position with numpy : 0.039206504821777344 nb_pixel_total : 15234 time to create 1 rle with old method : 0.0165255069732666 time for calcul the mask position with numpy : 0.03780674934387207 nb_pixel_total : 65830 time to create 1 rle with old method : 0.06692695617675781 time for calcul the mask position with numpy : 0.039907217025756836 nb_pixel_total : 6651 time to create 1 rle with old method : 0.007327556610107422 time for calcul the mask position with numpy : 0.04082155227661133 nb_pixel_total : 91339 time to create 1 rle with old method : 0.09400629997253418 time for calcul the mask position with numpy : 0.04032492637634277 nb_pixel_total : 35911 time to create 1 rle with old method : 0.0367887020111084 time for calcul the mask position with numpy : 0.03962206840515137 nb_pixel_total : 82675 time to create 1 rle with old method : 0.08547306060791016 time for calcul the mask position with numpy : 0.04491567611694336 nb_pixel_total : 643350 time to create 1 rle with new method : 0.39044690132141113 time for calcul the mask position with numpy : 0.061120033264160156 nb_pixel_total : 2376 time to create 1 rle with old method : 0.004359722137451172 time for calcul the mask position with numpy : 0.038068532943725586 nb_pixel_total : 51254 time to create 1 rle with old method : 0.08197522163391113 create new chi : 4.3516526222229 time to delete rle : 0.003545999526977539 batch 1 Loaded 20 chid ids of type : 3726 Number RLEs to save : 15562 TO DO : save crop sub photo not yet done ! save time : 1.2178163528442383 nb_obj : 17 nb_hashtags : 4 time to prepare the origin masks : 7.081545352935791 time for calcul the mask position with numpy : 0.6892852783203125 nb_pixel_total : 5454900 time to create 1 rle with new method : 2.9661991596221924 time for calcul the mask position with numpy : 0.02721691131591797 nb_pixel_total : 4098 time to create 1 rle with old method : 0.0076406002044677734 time for calcul the mask position with numpy : 0.03375124931335449 nb_pixel_total : 475851 time to create 1 rle with new method : 1.0241343975067139 time for calcul the mask position with numpy : 0.025178909301757812 nb_pixel_total : 738 time to create 1 rle with old method : 0.001779794692993164 time for calcul the mask position with numpy : 0.028046607971191406 nb_pixel_total : 153476 time to create 1 rle with new method : 0.7083652019500732 time for calcul the mask position with numpy : 0.022847652435302734 nb_pixel_total : 65508 time to create 1 rle with old method : 0.07302618026733398 time for calcul the mask position with numpy : 0.021570205688476562 nb_pixel_total : 23863 time to create 1 rle with old method : 0.025861501693725586 time for calcul the mask position with numpy : 0.022274017333984375 nb_pixel_total : 139221 time to create 1 rle with old method : 0.1533677577972412 time for calcul the mask position with numpy : 0.023597240447998047 nb_pixel_total : 3058 time to create 1 rle with old method : 0.004385709762573242 time for calcul the mask position with numpy : 0.0281527042388916 nb_pixel_total : 550056 time to create 1 rle with new method : 0.9248909950256348 time for calcul the mask position with numpy : 0.028143882751464844 nb_pixel_total : 7371 time to create 1 rle with old method : 0.00797271728515625 time for calcul the mask position with numpy : 0.03628993034362793 nb_pixel_total : 1456 time to create 1 rle with old method : 0.002447843551635742 time for calcul the mask position with numpy : 0.03886723518371582 nb_pixel_total : 25771 time to create 1 rle with old method : 0.02987360954284668 time for calcul the mask position with numpy : 0.03382301330566406 nb_pixel_total : 114352 time to create 1 rle with old method : 0.14368343353271484 time for calcul the mask position with numpy : 0.0367887020111084 nb_pixel_total : 2878 time to create 1 rle with old method : 0.003418445587158203 time for calcul the mask position with numpy : 0.03897547721862793 nb_pixel_total : 1561 time to create 1 rle with old method : 0.0018303394317626953 time for calcul the mask position with numpy : 0.03549790382385254 nb_pixel_total : 18813 time to create 1 rle with old method : 0.025220632553100586 time for calcul the mask position with numpy : 0.03601217269897461 nb_pixel_total : 7269 time to create 1 rle with old method : 0.008456230163574219 create new chi : 7.469085931777954 time to delete rle : 0.002020597457885742 batch 1 Loaded 18 chid ids of type : 3726 Number RLEs to save : 15784 TO DO : save crop sub photo not yet done ! save time : 1.3744969367980957 nb_obj : 5 nb_hashtags : 2 time to prepare the origin masks : 2.5145010948181152 time for calcul the mask position with numpy : 0.044877052307128906 nb_pixel_total : 2570247 time to create 1 rle with new method : 0.18127799034118652 time for calcul the mask position with numpy : 0.02118086814880371 nb_pixel_total : 1721 time to create 1 rle with old method : 0.0020036697387695312 time for calcul the mask position with numpy : 0.022190332412719727 nb_pixel_total : 32449 time to create 1 rle with old method : 0.03537583351135254 time for calcul the mask position with numpy : 0.022956371307373047 nb_pixel_total : 153672 time to create 1 rle with new method : 0.1394951343536377 time for calcul the mask position with numpy : 0.022002458572387695 nb_pixel_total : 2464 time to create 1 rle with old method : 0.002894878387451172 time for calcul the mask position with numpy : 0.06091451644897461 nb_pixel_total : 4289687 time to create 1 rle with new method : 0.13863158226013184 create new chi : 0.7418899536132812 time to delete rle : 0.0007195472717285156 batch 1 Loaded 6 chid ids of type : 3726 Number RLEs to save : 7848 TO DO : save crop sub photo not yet done ! save time : 0.9759106636047363 map_output_result : {1335035145: (0.0, 'Should be the crop_list due to order', 0.0), 1335035141: (0.0, 'Should be the crop_list due to order', 0.0), 1335035136: (0.0, 'Should be the crop_list due to order', 0.0), 1335035120: (0.0, 'Should be the crop_list due to order', 0.0), 1335007135: (0.0, 'Should be the crop_list due to order', 0.0), 1335007108: (0.0, 'Should be the crop_list due to order', 0.0), 1335006903: (0.0, 'Should be the crop_list due to order', 0.0), 1335006844: (0.0, 'Should be the crop_list due to order', 0.0), 1335006842: (0.0, 'Should be the crop_list due to order', 0.0), 1335006840: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1335035145, 1335035141, 1335035136, 1335035120, 1335007135, 1335007108, 1335006903, 1335006844, 1335006842, 1335006840] Looping around the photos to save general results len do output : 10 /1335035145.Didn't retrieve data . /1335035141.Didn't retrieve data . /1335035136.Didn't retrieve data . /1335035120.Didn't retrieve data . /1335007135.Didn't retrieve data . /1335007108.Didn't retrieve data . /1335006903.Didn't retrieve data . /1335006844.Didn't retrieve data . /1335006842.Didn't retrieve data . /1335006840.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035145', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035141', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035136', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035120', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335007135', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335007108', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006903', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006844', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006842', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006840', None, None, None, None, None, '2557221') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.014005661010742188 save_final save missing photos in datou_result : time spend for datou_step_exec : 126.86114358901978 time spend to save output : 0.014641761779785156 total time spend for step 5 : 126.87578535079956 step6:crop_condition Thu Feb 6 02:20:38 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 170 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 4869462 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738804840_1838997 we have uploaded 1 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.7762720584869385 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 10 About to insert : list_path_to_insert length 10 new photo from crops ! About to upload 10 photos upload in portfolio : 4869462 init cache_photo without model_param we have 10 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738804848_1838997 we have uploaded 10 photos in the portfolio 4869462 time of upload the photos Elapsed time : 3.8120129108428955 we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738804854_1838997 we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.9489374160766602 we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 4869462 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738804858_1838997 we have uploaded 5 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.5144193172454834 we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! About to upload 8 photos upload in portfolio : 4869462 init cache_photo without model_param we have 8 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738804864_1838997 we have uploaded 8 photos in the portfolio 4869462 time of upload the photos Elapsed time : 4.032390356063843 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 4869462 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738804870_1838997 we have uploaded 4 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.5015859603881836 we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1335035145, 1335035141, 1335035136, 1335035120, 1335007135, 1335007108, 1335006903, 1335006844, 1335006842, 1335006840] Looping around the photos to save general results len do output : 31 /1335084073Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335084080Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335084082Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335084083Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335084084Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335084085Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335084086Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335084087Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335084088Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335084089Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335084090Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335084092Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335084093Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335084094Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335084096Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335084097Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335084099Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335084100Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335084101Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335084106Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335084107Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335084108Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335084109Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335084110Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335084111Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335084113Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335084114Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335084115Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335084116Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335084117Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335084118Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035145', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035141', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035136', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035120', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335007135', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335007108', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006903', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006844', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006842', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006840', None, None, None, None, None, '2557221') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 103 time used for this insertion : 0.29596686363220215 save_final save missing photos in datou_result : time spend for datou_step_exec : 37.90354776382446 time spend to save output : 0.2979292869567871 total time spend for step 6 : 38.20147705078125 step7:ventilate_hashtags_in_portfolio Thu Feb 6 02:21:16 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 20277551 get user id for portfolio 20277551 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20277551 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Carton_gris','Teint_Dans_La_Masse','kraft','environnement','Carton_brun','mal_croppe','metal','flou','papier','cartonnette','autre_refus','plastique')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20277551 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Carton_gris','Teint_Dans_La_Masse','kraft','environnement','Carton_brun','mal_croppe','metal','flou','papier','cartonnette','autre_refus','plastique')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20277551 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Carton_gris','Teint_Dans_La_Masse','kraft','environnement','Carton_brun','mal_croppe','metal','flou','papier','cartonnette','autre_refus','plastique')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/20278698,20278699,20278700,20278701,20278702,20278703,20278704,20278705,20278706,20278707,20278708,20278709?tags=Carton_gris,Teint_Dans_La_Masse,kraft,environnement,Carton_brun,mal_croppe,metal,flou,papier,cartonnette,autre_refus,plastique Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1335035145, 1335035141, 1335035136, 1335035120, 1335007135, 1335007108, 1335006903, 1335006844, 1335006842, 1335006840] Looping around the photos to save general results len do output : 1 /20277551. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035145', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035141', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035136', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035120', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335007135', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335007108', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006903', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006844', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006842', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006840', None, None, None, None, None, '2557221') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.01777815818786621 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.1202991008758545 time spend to save output : 0.021445274353027344 total time spend for step 7 : 4.141744375228882 step8:final Thu Feb 6 02:21:20 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1335035145: ('0.09609489658538523',), 1335035141: ('0.09609489658538523',), 1335035136: ('0.09609489658538523',), 1335035120: ('0.09609489658538523',), 1335007135: ('0.09609489658538523',), 1335007108: ('0.09609489658538523',), 1335006903: ('0.09609489658538523',), 1335006844: ('0.09609489658538523',), 1335006842: ('0.09609489658538523',), 1335006840: ('0.09609489658538523',)} new output for save of step final : {1335035145: ('0.09609489658538523',), 1335035141: ('0.09609489658538523',), 1335035136: ('0.09609489658538523',), 1335035120: ('0.09609489658538523',), 1335007135: ('0.09609489658538523',), 1335007108: ('0.09609489658538523',), 1335006903: ('0.09609489658538523',), 1335006844: ('0.09609489658538523',), 1335006842: ('0.09609489658538523',), 1335006840: ('0.09609489658538523',)} [1335035145, 1335035141, 1335035136, 1335035120, 1335007135, 1335007108, 1335006903, 1335006844, 1335006842, 1335006840] Looping around the photos to save general results len do output : 10 /1335035145.Didn't retrieve data . /1335035141.Didn't retrieve data . /1335035136.Didn't retrieve data . /1335035120.Didn't retrieve data . /1335007135.Didn't retrieve data . /1335007108.Didn't retrieve data . /1335006903.Didn't retrieve data . /1335006844.Didn't retrieve data . /1335006842.Didn't retrieve data . /1335006840.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035145', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035141', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035136', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035120', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335007135', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335007108', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006903', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006844', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006842', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006840', None, None, None, None, None, '2557221') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.014202356338500977 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.12021517753601074 time spend to save output : 0.016002655029296875 total time spend for step 8 : 0.13621783256530762 step9:velours_tree Thu Feb 6 02:21:20 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.15582633018493652 time spend to save output : 4.315376281738281e-05 total time spend for step 9 : 0.1558694839477539 step10:send_mail_cod Thu Feb 6 02:21:21 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P20277551_06-02-2025_02_21_21.pdf 20278700 imagette202787001738804881 20278703 imagette202787031738804881 20278704 imagette202787041738804881 20278705 imagette202787051738804881 20278707 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette202787071738804881 20278708 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette202787081738804881 20278709 change filename to text .change filename to text .change filename to text .change filename to text .imagette202787091738804882 20278698 change filename to text .change filename to text .change filename to text .imagette202786981738804882 20278699 change filename to text .imagette202786991738804882 20278702 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette202787021738804882 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20277551 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20278698,20278699,20278700,20278701,20278702,20278703,20278704,20278705,20278706,20278707,20278708,20278709?tags=Carton_gris,Teint_Dans_La_Masse,kraft,environnement,Carton_brun,mal_croppe,metal,flou,papier,cartonnette,autre_refus,plastique your option no_mail is active, we will not send the real mail to your client args[1335035145] : ('0.09609489658538523',) no score found for photo 1335035145 apple ('0.09609489658538523',) We are sending mail with results at report@fotonower.com args[1335035141] : ('0.09609489658538523',) no score found for photo 1335035141 apple ('0.09609489658538523',) We are sending mail with results at report@fotonower.com args[1335035136] : ('0.09609489658538523',) no score found for photo 1335035136 apple ('0.09609489658538523',) We are sending mail with results at report@fotonower.com args[1335035120] : ('0.09609489658538523',) no score found for photo 1335035120 apple ('0.09609489658538523',) We are sending mail with results at report@fotonower.com args[1335007135] : ('0.09609489658538523',) no score found for photo 1335007135 apple ('0.09609489658538523',) We are sending mail with results at report@fotonower.com args[1335007108] : ('0.09609489658538523',) no score found for photo 1335007108 apple ('0.09609489658538523',) We are sending mail with results at report@fotonower.com args[1335006903] : ('0.09609489658538523',) no score found for photo 1335006903 apple ('0.09609489658538523',) We are sending mail with results at report@fotonower.com args[1335006844] : ('0.09609489658538523',) no score found for photo 1335006844 apple ('0.09609489658538523',) We are sending mail with results at report@fotonower.com args[1335006842] : ('0.09609489658538523',) no score found for photo 1335006842 apple ('0.09609489658538523',) We are sending mail with results at report@fotonower.com args[1335006840] : ('0.09609489658538523',) no score found for photo 1335006840 apple ('0.09609489658538523',) We are sending mail with results at report@fotonower.com refus_total : 0.09609489658538523 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20277551 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1335006844,1335006903,1335007135,1335035120,1335035136,1335035141,1335035145,1335006840,1335006842,1335007108) Found this number of photos: 10 begin to download photo : 1335006844 begin to download photo : 1335035120 begin to download photo : 1335035145 begin to download photo : 1335007108 download finish for photo 1335035120 begin to download photo : 1335035136 download finish for photo 1335007108 download finish for photo 1335006844 begin to download photo : 1335006903 download finish for photo 1335035145 begin to download photo : 1335006840 download finish for photo 1335035136 begin to download photo : 1335035141 download finish for photo 1335006903 begin to download photo : 1335007135 download finish for photo 1335006840 begin to download photo : 1335006842 download finish for photo 1335035141 download finish for photo 1335006842 download finish for photo 1335007135 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20277551_06-02-2025_02_21_21.pdf results_Auto_P20277551_06-02-2025_02_21_21.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20277551_06-02-2025_02_21_21.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3459','20277551','results_Auto_P20277551_06-02-2025_02_21_21.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20277551_06-02-2025_02_21_21.pdf','pdf','','0.59','0.09609489658538523') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1335035145, 1335035141, 1335035136, 1335035120, 1335007135, 1335007108, 1335006903, 1335006844, 1335006842, 1335006840] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035145', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035141', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035136', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035120', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335007135', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335007108', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006903', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006844', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006842', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006840', None, None, None, None, None, '2557221') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.013513326644897461 save_final save missing photos in datou_result : time spend for datou_step_exec : 5.018271446228027 time spend to save output : 0.014185905456542969 total time spend for step 10 : 5.03245735168457 step11:split_time_score Thu Feb 6 02:21:26 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('18', 8), ('19', 2)) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 05022025 20277551 Nombre de photos uploadées : 10 / 23040 (0%) 05022025 20277551 Nombre de photos taguées (types de déchets): 0 / 10 (0%) 05022025 20277551 Nombre de photos taguées (volume) : 0 / 10 (0%) elapsed_time : load_data_split_time_score 4.0531158447265625e-06 elapsed_time : order_list_meta_photo_and_scores 1.2874603271484375e-05 ?????????? elapsed_time : fill_and_build_computed_from_old_data 0.0013532638549804688 elapsed_time : insert_dashboard_record_day_entry 0.023970842361450195 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277499 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277500 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277501 order by id desc limit 1 Qualite : 0.22049059616263506 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277502 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20277502 AND mptpi.`type`=3594 To do Qualite : 0.24811212809776687 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20277503_06-02-2025_02_17_05.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277503 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20277503 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277504 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277505 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277506 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277507 order by id desc limit 1 Qualite : 0.06442282072148928 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20277508_06-02-2025_00_55_54.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277508 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20277508 AND mptpi.`type`=3726 To do Qualite : 0.2215438765205157 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20277509_06-02-2025_01_49_54.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277509 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20277509 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277510 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277511 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277512 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277513 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277514 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277515 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277516 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277517 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277518 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277519 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20277520_06-02-2025_02_14_45.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277520 order by id desc limit 1 Qualite : 0.11579544015203329 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20277521_06-02-2025_01_31_22.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277521 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20277521 AND mptpi.`type`=3327 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277522 order by id desc limit 1 Qualite : 0.1770747284253208 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20277523_06-02-2025_01_29_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277523 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20277523 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277524 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277525 order by id desc limit 1 Qualite : 0.07058148054050647 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20277526_06-02-2025_02_11_55.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277526 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20277526 AND mptpi.`type`=4207 To do Qualite : 0.06769655641860853 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20277527_06-02-2025_00_59_31.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277527 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20277527 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277528 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277529 order by id desc limit 1 Qualite : 0.21480488134682896 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20277530_06-02-2025_01_27_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277530 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20277530 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277531 order by id desc limit 1 Qualite : 0.15524193513134887 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20277532_06-02-2025_01_22_04.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277532 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20277532 AND mptpi.`type`=3327 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277533 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277534 order by id desc limit 1 Qualite : 0.09479940201243531 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20277535_06-02-2025_02_06_44.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277535 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20277535 AND mptpi.`type`=4207 To do Qualite : 0.05092112302991469 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20277548_06-02-2025_02_15_41.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277548 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20277548 AND mptpi.`type`=4207 To do Qualite : 0.180145604972313 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20277549_06-02-2025_00_58_40.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277549 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20277549 AND mptpi.`type`=3594 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20277550_06-02-2025_01_20_52.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277550 order by id desc limit 1 Qualite : 0.09609489658538523 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20277551_06-02-2025_02_21_21.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277551 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20277551 AND mptpi.`type`=3726 To do Qualite : 0.06784738675129119 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20277552_06-02-2025_01_13_42.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20277552 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20277552 AND mptpi.`type`=4207 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'05022025': {'nb_upload': 10, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1335035145, 1335035141, 1335035136, 1335035120, 1335007135, 1335007108, 1335006903, 1335006844, 1335006842, 1335006840] Looping around the photos to save general results len do output : 1 /20277551Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035145', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035141', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035136', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335035120', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335007135', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335007108', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006903', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006844', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006842', None, None, None, None, None, '2557221') ('3459', None, None, None, None, None, None, None, '2557221') ('3459', '20277551', '1335006840', None, None, None, None, None, '2557221') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.014840126037597656 save_final save missing photos in datou_result : time spend for datou_step_exec : 21.70396089553833 time spend to save output : 0.01513361930847168 total time spend for step 11 : 21.7190945148468 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 1 len(input) + len(total_photo_id_missing) : 10 set_done_treatment 165.08user 102.55system 5:23.90elapsed 82%CPU (0avgtext+0avgdata 4531308maxresident)k 115656inputs+123032outputs (6649major+10407347minor)pagefaults 0swaps