python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' -a 4741 -l 30 ' -s datou_current_4741 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 576256 load datou : 4741 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5732, 'mask_refus_amiens_050123', 16384, 25088, 'mask_refus_amiens_050123', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 1, 5, 13, 23, 59), datetime.datetime(2023, 1, 5, 13, 23, 59)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5972, 'learn_entrant_syctomXV_111023', 16384, 25088, 'learn_entrant_syctomXV_111023', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 10, 11, 15, 57, 35), datetime.datetime(2023, 10, 11, 15, 57, 35)) load thcls load THCL from format json or kwargs add thcl : 3453 in CacheModelConfig load THCL from format json or kwargs add thcl : 3783 in CacheModelConfig load pdts add pdt : 5732 in CacheModelConfig add pdt : 5972 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4741, datou_cur_ids : ['2797324'] with mtr_portfolio_ids : ['22713351'] and first list_photo_ids : [] new path : /proc/576256/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, mask_detect, blur_detection, brightness, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , BFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 3 ; length of list_pids : 3 ; length of list_args : 3 time to download the photos : 0.7607278823852539 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Tue May 6 11:41:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 5842 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-05-06 11:41:32.406730: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-05-06 11:41:32.431243: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-05-06 11:41:32.433873: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f4f78000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-05-06 11:41:32.433934: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-05-06 11:41:32.438210: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-05-06 11:41:32.568962: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x12512f70 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-05-06 11:41:32.569023: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-05-06 11:41:32.570250: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-06 11:41:32.571642: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 11:41:32.577201: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 11:41:32.580653: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-06 11:41:32.582096: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-06 11:41:32.586647: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-06 11:41:32.588355: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-06 11:41:32.595024: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 11:41:32.596473: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-06 11:41:32.596551: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 11:41:32.597305: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-06 11:41:32.597323: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-06 11:41:32.597334: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-06 11:41:32.598692: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 5349 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-05-06 11:41:32.956465: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-06 11:41:32.956598: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 11:41:32.956623: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 11:41:32.956645: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-06 11:41:32.956665: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-06 11:41:32.956686: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-06 11:41:32.956705: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-06 11:41:32.956726: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 11:41:32.957939: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-06 11:41:32.959324: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-06 11:41:32.959365: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 11:41:32.959384: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 11:41:32.959400: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-06 11:41:32.959430: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-06 11:41:32.959447: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-06 11:41:32.959463: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-06 11:41:32.959479: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 11:41:32.960677: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-06 11:41:32.960719: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-06 11:41:32.960730: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-06 11:41:32.960740: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-06 11:41:32.962004: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 5349 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl3783 thcls : [{'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5972 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5972, 'learn_entrant_syctomXV_111023', 16384, 25088, 'learn_entrant_syctomXV_111023', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 10, 11, 15, 57, 35), datetime.datetime(2023, 10, 11, 15, 57, 35)) {'thcl': {'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'sac', 'textiles', 'verre', 'organique', 'dasri', 'masque', 'encombrant', 'autre_emballage', 'autre_non_emballage', 'environnement'], 'list_hashtags_csv': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'svm_hashtag_type_desc': 5972, 'photo_desc_type': 5972, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'sac', 'textiles', 'verre', 'organique', 'dasri', 'masque', 'encombrant', 'autre_emballage', 'autre_non_emballage', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_entrant_syctomXV_111023 NUM_CLASSES 11 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_entrant_syctomXV_111023 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-05-06 11:41:43.312730: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 11:41:43.476225: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_entrant_syctomXV_111023 /data/models_weight/learn_entrant_syctomXV_111023/mask_model.h5 size_local : 256052544 size in s3 : 256052544 create time local : 2023-10-26 16:36:43 create time in s3 : 2023-10-11 13:57:17 mask_model.h5 already exist and didn't need to update list_images length : 3 NEW PHOTO Processing 1 images image shape: (3120, 4160, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 14 NEW PHOTO Processing 1 images image shape: (3120, 4160, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 16 NEW PHOTO Processing 1 images image shape: (3120, 4160, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 138.20000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 2 Detection mask done ! Trying to reset tf kernel 576557 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 553 tf kernel not reseted sub process len(results) : 3 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 3 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5842 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl3783 Catched exception ! Connect or reconnect ! thcls : [{'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5972 ['background', 'sac', 'textiles', 'verre', 'organique', 'dasri', 'masque', 'encombrant', 'autre_emballage', 'autre_non_emballage', 'environnement'] time for calcul the mask position with numpy : 0.018939495086669922 nb_pixel_total : 459665 time to create 1 rle with new method : 0.042671918869018555 length of segment : 855 time for calcul the mask position with numpy : 0.0017342567443847656 nb_pixel_total : 88966 time to create 1 rle with old method : 0.10355234146118164 length of segment : 467 time for calcul the mask position with numpy : 0.000621795654296875 nb_pixel_total : 38657 time to create 1 rle with old method : 0.04391026496887207 length of segment : 227 time for calcul the mask position with numpy : 0.00014829635620117188 nb_pixel_total : 4853 time to create 1 rle with old method : 0.00583338737487793 length of segment : 92 time for calcul the mask position with numpy : 0.00038909912109375 nb_pixel_total : 13837 time to create 1 rle with old method : 0.016906261444091797 length of segment : 114 time for calcul the mask position with numpy : 0.0009188652038574219 nb_pixel_total : 40404 time to create 1 rle with old method : 0.047069549560546875 length of segment : 322 time for calcul the mask position with numpy : 0.0007786750793457031 nb_pixel_total : 31872 time to create 1 rle with old method : 0.03753352165222168 length of segment : 137 time for calcul the mask position with numpy : 0.0010938644409179688 nb_pixel_total : 64804 time to create 1 rle with old method : 0.07606697082519531 length of segment : 386 time for calcul the mask position with numpy : 0.0005340576171875 nb_pixel_total : 22644 time to create 1 rle with old method : 0.02531719207763672 length of segment : 155 time for calcul the mask position with numpy : 0.0018157958984375 nb_pixel_total : 88604 time to create 1 rle with old method : 0.10519146919250488 length of segment : 357 time for calcul the mask position with numpy : 0.0007104873657226562 nb_pixel_total : 33918 time to create 1 rle with old method : 0.04687166213989258 length of segment : 201 time for calcul the mask position with numpy : 0.0007979869842529297 nb_pixel_total : 36923 time to create 1 rle with old method : 0.04260087013244629 length of segment : 217 time for calcul the mask position with numpy : 0.0016868114471435547 nb_pixel_total : 70115 time to create 1 rle with old method : 0.07922744750976562 length of segment : 421 time for calcul the mask position with numpy : 0.0004000663757324219 nb_pixel_total : 13963 time to create 1 rle with old method : 0.01663684844970703 length of segment : 143 time for calcul the mask position with numpy : 0.005979776382446289 Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl3453 thcls : [{'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5732 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5732, 'mask_refus_amiens_050123', 16384, 25088, 'mask_refus_amiens_050123', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 1, 5, 13, 23, 59), datetime.datetime(2023, 1, 5, 13, 23, 59)) {'thcl': {'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'svm_hashtag_type_desc': 5732, 'photo_desc_type': 5732, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME mask_refus_amiens_050123 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : mask_refus_amiens_050123 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-05-06 11:42:21.616536: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 11:42:21.816161: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/mask_refus_amiens_050123 /data/models_weight/mask_refus_amiens_050123/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2023-01-05 13:44:07 create time in s3 : 2023-01-05 12:23:52 mask_model.h5 already exist and didn't need to update list_images length : 3 NEW PHOTO Processing 1 images image shape: (3120, 4160, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 72 NEW PHOTO Processing 1 images image shape: (3120, 4160, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 54 NEW PHOTO Processing 1 images image shape: (3120, 4160, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 138.20000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 4 Detection mask done ! Trying to reset tf kernel 577932 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1746 tf kernel not reseted sub process len(results) : 3 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 3 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 7035 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl3453 Catched exception ! Connect or reconnect ! thcls : [{'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5732 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0025632381439208984 nb_pixel_total : 61530 time to create 1 rle with old method : 0.1061558723449707 length of segment : 181 time for calcul the mask position with numpy : 0.0012543201446533203 nb_pixel_total : 14095 time to create 1 rle with old method : 0.02585577964782715 length of segment : 397 time for calcul the mask position with numpy : 0.004725217819213867 nb_pixel_total : 76470 time to create 1 rle with old method : 0.10480308532714844 length of segment : 324 time for calcul the mask position with numpy : 0.0022857189178466797 nb_pixel_total : 48483 time to create 1 rle with old method : 0.05591940879821777 length of segment : 322 time for calcul the mask position with numpy : 0.004934787750244141 nb_pixel_total : 114126 time to create 1 rle with old method : 0.15353608131408691 length of segment : 512 time for calcul the mask position with numpy : 0.003348112106323242 nb_pixel_total : 85574 time to create 1 rle with old method : 0.11652588844299316 length of segment : 439 time for calcul the mask position with numpy : 0.0010612010955810547 nb_pixel_total : 40144 time to create 1 rle with old method : 0.0464930534362793 length of segment : 154 time for calcul the mask position with numpy : 0.0013055801391601562 nb_pixel_total : 57938 time to create 1 rle with old method : 0.0666208267211914 length of segment : 259 time for calcul the mask position with numpy : 0.0013942718505859375 nb_pixel_total : 31946 time to create 1 rle with old method : 0.037042856216430664 length of segment : 245 time for calcul the mask position with numpy : 0.018181324005126953 nb_pixel_total : 724652 time to create 1 rle with new method : 0.08615732192993164 length of segment : 1034 time for calcul the mask position with numpy : 0.0071675777435302734 nb_pixel_total : 425489 time to create 1 rle with new method : 0.02335500717163086 length of segment : 619 time for calcul the mask position with numpy : 0.0013492107391357422 nb_pixel_total : 54243 time to create 1 rle with old method : 0.059689998626708984 length of segment : 258 time for calcul the mask position with numpy : 0.00639033317565918 nb_pixel_total : 340348 time to create 1 rle with new method : 0.01987767219543457 length of segment : 754 time for calcul the mask position with numpy : 0.0008544921875 nb_pixel_total : 38553 time to create 1 rle with old method : 0.04251742362976074 length of segment : 279 time for calcul the mask position with numpy : 0.0006506443023681641 nb_pixel_total : 22578 time to create 1 rle with old method : 0.02607107162475586 length of segment : 141 time for calcul the mask position with numpy : 0.004256486892700195 nb_pixel_total : 276585 time to create 1 rle with new method : 0.010720014572143555 length of segment : 455 time for calcul the mask position with numpy : 0.0012433528900146484 nb_pixel_total : 49890 time to create 1 rle with old method : 0.056143760681152344 length of segment : 267 time for calcul the mask position with numpy : 0.0024499893188476562 nb_pixel_total : 107838 time to create 1 rle with old method : 0.12497234344482422 length of segment : 446 time for calcul the mask position with numpy : 0.0014545917510986328 nb_pixel_total : 53434 time to create 1 rle with old method : 0.07348322868347168 length of segment : 262 time for calcul the mask position with numpy : 0.014905929565429688 nb_pixel_total : 527236 time to create 1 rle with new method : 0.1972062587738037 length of segment : 948 time for calcul the mask position with numpy : 0.026366472244262695 nb_pixel_total : 1124799 time to create 1 rle with new method : 0.23314690589904785 length of segment : 1031 time for calcul the mask position with numpy : 0.0010135173797607422 nb_pixel_total : 9519 time to create 1 rle with old method : 0.012159585952758789 length of segment : 122 time for calcul the mask position with numpy : 0.0015101432800292969 nb_pixel_total : 56753 time to create 1 rle with old method : 0.06728744506835938 length of segment : 325 time for calcul the mask position with numpy : 0.0016732215881347656 nb_pixel_total : 56788 time to create 1 rle with old method : 0.06476354598999023 length of segment : 541 time for calcul the mask position with numpy : 0.002009868621826172 nb_pixel_total : 86021 time to create 1 rle with old method : 0.10066556930541992 length of segment : 357 time for calcul the mask position with numpy : 0.0030150413513183594 nb_pixel_total : 72060 time to create 1 rle with old method : 0.11349987983703613 length of segment : 319 time for calcul the mask position with numpy : 0.0020279884338378906 nb_pixel_total : 42894 time to create 1 rle with old method : 0.05982637405395508 length of segment : 202 time for calcul the mask position with numpy : 0.0014986991882324219 nb_pixel_total : 46872 time to create 1 rle with old method : 0.06557869911193848 length of segment : 324 time for calcul the mask position with numpy : 0.003983974456787109 nb_pixel_total : 84655 time to create 1 rle with old method : 0.10101699829101562 length of segment : 561 time for calcul the mask position with numpy : 0.0017938613891601562 nb_pixel_total : 2269 time to create 1 rle with old method : 0.003345012664794922 length of segment : 136 time for calcul the mask position with numpy : 0.0003345012664794922 nb_pixel_total : 2825 time to create 1 rle with old method : 0.0056951045989990234 length of segment : 149 time for calcul the mask position with numpy : 0.006654500961303711 nb_pixel_total : 73102 time to create 1 rle with old method : 0.10332632064819336 length of segment : 277 time for calcul the mask position with numpy : 0.0016865730285644531 nb_pixel_total : 65175 time to create 1 rle with old method : 0.08765888214111328 length of segment : 262 time for calcul the mask position with numpy : 0.003705263137817383 nb_pixel_total : 139634 time to create 1 rle with old method : 0.16360735893249512 length of segment : 484 time for calcul the mask position with numpy : 0.0012269020080566406 nb_pixel_total : 57943 time to create 1 rle with old method : 0.0651712417602539 length of segment : 282 time for calcul the mask position with numpy : 0.003721952438354492 nb_pixel_total : 157577 time to create 1 rle with new method : 0.009372711181640625 length of segment : 414 time for calcul the mask position with numpy : 0.0001761913299560547 nb_pixel_total : 5228 time to create 1 rle with old method : 0.006158113479614258 length of segment : 95 time for calcul the mask position with numpy : 0.005277156829833984 nb_pixel_total : 238104 time to create 1 rle with new method : 0.012577295303344727 length of segment : 588 time for calcul the mask position with numpy : 0.010775566101074219 nb_pixel_total : 337501 time to create 1 rle with new method : 0.026889562606811523 length of segment : 803 time for calcul the mask position with numpy : 0.0004677772521972656 nb_pixel_total : 16473 time to create 1 rle with old method : 0.019156932830810547 length of segment : 108 time for calcul the mask position with numpy : 0.0010595321655273438 nb_pixel_total : 43919 time to create 1 rle with old method : 0.050228118896484375 length of segment : 283 time for calcul the mask position with numpy : 0.0017364025115966797 nb_pixel_total : 66529 time to create 1 rle with old method : 0.07528305053710938 length of segment : 352 time for calcul the mask position with numpy : 0.0006973743438720703 nb_pixel_total : 28168 time to create 1 rle with old method : 0.03254413604736328 length of segment : 198 time for calcul the mask position with numpy : 0.0012738704681396484 nb_pixel_total : 33265 time to create 1 rle with old method : 0.038103342056274414 length of segment : 288 time for calcul the mask position with numpy : 0.0007245540618896484 nb_pixel_total : 14368 time to create 1 rle with old method : 0.016777515411376953 length of segment : 181 time for calcul the mask position with numpy : 0.0032203197479248047 nb_pixel_total : 143086 time to create 1 rle with old method : 0.1626420021057129 length of segment : 425 time for calcul the mask position with numpy : 0.005417585372924805 nb_pixel_total : 246436 time to create 1 rle with new method : 0.013277292251586914 length of segment : 824 time for calcul the mask position with numpy : 0.0034453868865966797 nb_pixel_total : 130560 time to create 1 rle with old method : 0.14663290977478027 length of segment : 464 time for calcul the mask position with numpy : 0.0018062591552734375 nb_pixel_total : 66758 time to create 1 rle with old method : 0.07503962516784668 length of segment : 331 time for calcul the mask position with numpy : 0.005025148391723633 nb_pixel_total : 139153 time to create 1 rle with old method : 0.16034150123596191 length of segment : 531 time for calcul the mask position with numpy : 0.0018315315246582031 nb_pixel_total : 76754 time to create 1 rle with old method : 0.08675527572631836 length of segment : 348 time for calcul the mask position with numpy : 0.001544952392578125 nb_pixel_total : 61074 time to create 1 rle with old method : 0.07047700881958008 length of segment : 283 time for calcul the mask position with numpy : 0.01123189926147461 nb_pixel_total : 469732 time to create 1 rle with new method : 0.04631853103637695 length of segment : 1096 time for calcul the mask position with numpy : 0.0015897750854492188 nb_pixel_total : 66285 time to create 1 rle with old method : 0.0765070915222168 length of segment : 315 time for calcul the mask position with numpy : 0.0014657974243164062 nb_pixel_total : 9892 time to create 1 rle with old method : 0.012725591659545898 length of segment : 112 time for calcul the mask position with numpy : 0.0027904510498046875 nb_pixel_total : 121825 time to create 1 rle with old method : 0.14256048202514648 length of segment : 327 time for calcul the mask position with numpy : 0.002468109130859375 nb_pixel_total : 110712 time to create 1 rle with old method : 0.12682342529296875 length of segment : 410 time for calcul the mask position with numpy : 0.001512289047241211 nb_pixel_total : 63471 time to create 1 rle with old method : 0.07275819778442383 length of segment : 252 time for calcul the mask position with numpy : 0.0006136894226074219 nb_pixel_total : 30579 time to create 1 rle with old method : 0.03541731834411621 length of segment : 146 time for calcul the mask position with numpy : 0.002280712127685547 nb_pixel_total : 103834 time to create 1 rle with old method : 0.15357589721679688 length of segment : 517 time for calcul the mask position with numpy : 0.009860038757324219 nb_pixel_total : 462032 time to create 1 rle with new method : 0.024626970291137695 length of segment : 1195 time for calcul the mask position with numpy : 0.002275228500366211 nb_pixel_total : 49101 time to create 1 rle with old method : 0.05641007423400879 length of segment : 440 time for calcul the mask position with numpy : 0.0021452903747558594 nb_pixel_total : 82931 time to create 1 rle with old method : 0.09296250343322754 length of segment : 331 time for calcul the mask position with numpy : 0.0020143985748291016 nb_pixel_total : 87620 time to create 1 rle with old method : 0.09856986999511719 length of segment : 504 time for calcul the mask position with numpy : 0.001954793930053711 nb_pixel_total : 74119 time to create 1 rle with old method : 0.08359026908874512 length of segment : 266 time for calcul the mask position with numpy : 0.0070416927337646484 nb_pixel_total : 229209 time to create 1 rle with new method : 0.019615650177001953 length of segment : 609 time for calcul the mask position with numpy : 0.0018436908721923828 nb_pixel_total : 92165 time to create 1 rle with old method : 0.10243463516235352 length of segment : 537 time for calcul the mask position with numpy : 0.0011768341064453125 nb_pixel_total : 58495 time to create 1 rle with old method : 0.09610319137573242 length of segment : 253 time for calcul the mask position with numpy : 0.0037431716918945312 nb_pixel_total : 150829 time to create 1 rle with new method : 0.011395454406738281 length of segment : 652 time for calcul the mask position with numpy : 0.01588273048400879 nb_pixel_total : 391438 time to create 1 rle with new method : 0.05613350868225098 length of segment : 1190 time for calcul the mask position with numpy : 0.0017664432525634766 nb_pixel_total : 68243 time to create 1 rle with old method : 0.09293246269226074 length of segment : 356 time for calcul the mask position with numpy : 0.0029783248901367188 nb_pixel_total : 159360 time to create 1 rle with new method : 0.007645368576049805 length of segment : 453 time for calcul the mask position with numpy : 0.0021355152130126953 nb_pixel_total : 122314 time to create 1 rle with old method : 0.13955211639404297 length of segment : 318 time for calcul the mask position with numpy : 0.008203983306884766 nb_pixel_total : 303230 time to create 1 rle with new method : 0.04449582099914551 length of segment : 760 time for calcul the mask position with numpy : 0.0011997222900390625 nb_pixel_total : 56952 time to create 1 rle with old method : 0.0662841796875 length of segment : 205 time for calcul the mask position with numpy : 0.0008208751678466797 nb_pixel_total : 32219 time to create 1 rle with old method : 0.03768610954284668 length of segment : 231 time for calcul the mask position with numpy : 0.0009207725524902344 nb_pixel_total : 33942 time to create 1 rle with old method : 0.03955817222595215 length of segment : 240 time for calcul the mask position with numpy : 0.0020055770874023438 nb_pixel_total : 76511 time to create 1 rle with old method : 0.0867164134979248 length of segment : 536 time for calcul the mask position with numpy : 0.0011477470397949219 nb_pixel_total : 56477 time to create 1 rle with old method : 0.07871294021606445 length of segment : 210 time for calcul the mask position with numpy : 0.0020551681518554688 nb_pixel_total : 94199 time to create 1 rle with old method : 0.10649251937866211 length of segment : 445 time for calcul the mask position with numpy : 0.0009014606475830078 nb_pixel_total : 34664 time to create 1 rle with old method : 0.03967928886413574 length of segment : 218 time for calcul the mask position with numpy : 0.0019440650939941406 nb_pixel_total : 85694 time to create 1 rle with old method : 0.09795427322387695 length of segment : 353 time for calcul the mask position with numpy : 0.0006701946258544922 nb_pixel_total : 12136 time to create 1 rle with old method : 0.013926982879638672 length of segment : 310 time for calcul the mask position with numpy : 0.0007212162017822266 nb_pixel_total : 34145 time to create 1 rle with old method : 0.039357662200927734 length of segment : 244 time for calcul the mask position with numpy : 0.0015447139739990234 nb_pixel_total : 87314 time to create 1 rle with old method : 0.0967257022857666 length of segment : 333 time for calcul the mask position with numpy : 0.0005936622619628906 nb_pixel_total : 15030 time to create 1 rle with old method : 0.016781091690063477 length of segment : 329 time for calcul the mask position with numpy : 0.0056722164154052734 nb_pixel_total : 161217 time to create 1 rle with new method : 0.015098333358764648 length of segment : 641 time for calcul the mask position with numpy : 0.012104988098144531 nb_pixel_total : 448260 time to create 1 rle with new method : 0.028449058532714844 length of segment : 644 time for calcul the mask position with numpy : 0.0008606910705566406 nb_pixel_total : 30932 time to create 1 rle with old method : 0.0352482795715332 length of segment : 246 time for calcul the mask position with numpy : 0.008134126663208008 nb_pixel_total : 80041 time to create 1 rle with old method : 0.08801579475402832 length of segment : 1817 time for calcul the mask position with numpy : 0.008019447326660156 nb_pixel_total : 365119 time to create 1 rle with new method : 0.022049427032470703 length of segment : 642 time for calcul the mask position with numpy : 0.003997087478637695 nb_pixel_total : 175507 time to create 1 rle with new method : 0.009943008422851562 length of segment : 481 time for calcul the mask position with numpy : 0.007351875305175781 nb_pixel_total : 301086 time to create 1 rle with new method : 0.019052505493164062 length of segment : 751 time for calcul the mask position with numpy : 0.0004100799560546875 nb_pixel_total : 14252 time to create 1 rle with old method : 0.016382694244384766 length of segment : 154 time for calcul the mask position with numpy : 0.0005056858062744141 nb_pixel_total : 15429 time to create 1 rle with old method : 0.01805877685546875 length of segment : 91 time for calcul the mask position with numpy : 0.0006229877471923828 nb_pixel_total : 9872 time to create 1 rle with old method : 0.011737823486328125 length of segment : 324 time for calcul the mask position with numpy : 0.008118629455566406 nb_pixel_total : 412536 time to create 1 rle with new method : 0.018761157989501953 length of segment : 592 time for calcul the mask position with numpy : 0.0006799697875976562 nb_pixel_total : 29519 time to create 1 rle with old method : 0.03195953369140625 length of segment : 238 time for calcul the mask position with numpy : 0.0008955001831054688 nb_pixel_total : 43512 time to create 1 rle with old method : 0.0476377010345459 length of segment : 207 time for calcul the mask position with numpy : 0.012948036193847656 nb_pixel_total : 298753 time to create 1 rle with new method : 0.5617048740386963 length of segment : 758 time for calcul the mask position with numpy : 0.002183675765991211 nb_pixel_total : 123923 time to create 1 rle with old method : 0.13200855255126953 length of segment : 484 time for calcul the mask position with numpy : 0.003992795944213867 nb_pixel_total : 148177 time to create 1 rle with old method : 0.169677734375 length of segment : 713 time for calcul the mask position with numpy : 0.0010256767272949219 nb_pixel_total : 43540 time to create 1 rle with old method : 0.04630303382873535 length of segment : 200 time for calcul the mask position with numpy : 0.0004420280456542969 nb_pixel_total : 23552 time to create 1 rle with old method : 0.02491927146911621 length of segment : 249 time for calcul the mask position with numpy : 0.00788116455078125 nb_pixel_total : 330961 time to create 1 rle with new method : 0.034705162048339844 length of segment : 701 time for calcul the mask position with numpy : 0.003935337066650391 nb_pixel_total : 200391 time to create 1 rle with new method : 0.011652469635009766 length of segment : 987 time for calcul the mask position with numpy : 0.003935575485229492 nb_pixel_total : 259956 time to create 1 rle with new method : 0.009930849075317383 length of segment : 555 time for calcul the mask position with numpy : 0.0005095005035400391 nb_pixel_total : 20373 time to create 1 rle with old method : 0.02229928970336914 length of segment : 186 time for calcul the mask position with numpy : 0.003937721252441406 nb_pixel_total : 153390 time to create 1 rle with new method : 0.012107372283935547 length of segment : 577 time for calcul the mask position with numpy : 0.002906322479248047 nb_pixel_total : 105640 time to create 1 rle with old method : 0.1126863956451416 length of segment : 489 time for calcul the mask position with numpy : 0.002534627914428711 nb_pixel_total : 100593 time to create 1 rle with old method : 0.1351146697998047 length of segment : 402 time for calcul the mask position with numpy : 0.0005626678466796875 nb_pixel_total : 22009 time to create 1 rle with old method : 0.024554729461669922 length of segment : 183 time for calcul the mask position with numpy : 0.0013780593872070312 nb_pixel_total : 75760 time to create 1 rle with old method : 0.080596923828125 length of segment : 332 time for calcul the mask position with numpy : 0.0028641223907470703 nb_pixel_total : 127508 time to create 1 rle with old method : 0.1488039493560791 length of segment : 550 time for calcul the mask position with numpy : 0.002492189407348633 nb_pixel_total : 122877 time to create 1 rle with old method : 0.13169598579406738 length of segment : 380 time for calcul the mask position with numpy : 0.0052738189697265625 nb_pixel_total : 78839 time to create 1 rle with old method : 0.08548665046691895 length of segment : 869 time for calcul the mask position with numpy : 0.0013148784637451172 nb_pixel_total : 66774 time to create 1 rle with old method : 0.07051968574523926 length of segment : 299 time for calcul the mask position with numpy : 0.0038950443267822266 nb_pixel_total : 254857 time to create 1 rle with new method : 0.009915590286254883 length of segment : 557 time for calcul the mask position with numpy : 0.0018765926361083984 nb_pixel_total : 77619 time to create 1 rle with old method : 0.08274483680725098 length of segment : 550 time for calcul the mask position with numpy : 0.001580953598022461 nb_pixel_total : 69144 time to create 1 rle with old method : 0.07467389106750488 length of segment : 271 time for calcul the mask position with numpy : 0.0022652149200439453 nb_pixel_total : 87268 time to create 1 rle with old method : 0.10911154747009277 length of segment : 412 time for calcul the mask position with numpy : 0.01868915557861328 nb_pixel_total : 587632 time to create 1 rle with new method : 0.6809146404266357 length of segment : 875 time for calcul the mask position with numpy : 0.001062154769897461 nb_pixel_total : 46164 time to create 1 rle with old method : 0.06263852119445801 length of segment : 319 time for calcul the mask position with numpy : 0.001528024673461914 nb_pixel_total : 87474 time to create 1 rle with old method : 0.09307742118835449 length of segment : 321 time for calcul the mask position with numpy : 0.0027556419372558594 nb_pixel_total : 126789 time to create 1 rle with old method : 0.13472557067871094 length of segment : 391 time for calcul the mask position with numpy : 0.0027875900268554688 nb_pixel_total : 114923 time to create 1 rle with old method : 0.1387310028076172 length of segment : 469 time for calcul the mask position with numpy : 0.04386734962463379 nb_pixel_total : 1403790 time to create 1 rle with new method : 0.6445059776306152 length of segment : 2520 time for calcul the mask position with numpy : 0.021048307418823242 nb_pixel_total : 1133214 time to create 1 rle with new method : 0.17462801933288574 length of segment : 2173 time for calcul the mask position with numpy : 0.019138336181640625 nb_pixel_total : 1537937 time to create 1 rle with new method : 0.10571146011352539 length of segment : 1271 time for calcul the mask position with numpy : 0.0021321773529052734 nb_pixel_total : 166749 time to create 1 rle with new method : 0.0043523311614990234 length of segment : 450 time spent for convertir_results : 23.18046498298645 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 162 chid ids of type : 4854 +++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 0 save missing photos in datou_result : time spend for datou_step_exec : 78.08488869667053 time spend to save output : 0.12539362907409668 total time spend for step 2 : 78.21028232574463 step3:blur_detection Tue May 6 11:43:23 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: variance laplacian (3120, 4160) 3216.375394352539 (3120, 4160) 4146.6991381408625 (3120, 4160) 180.04659657686707 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 3 time used for this insertion : 0.01500558853149414 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 3 time used for this insertion : 0.010314464569091797 save missing photos in datou_result : time spend for datou_step_exec : 1.8866701126098633 time spend to save output : 0.029725313186645508 total time spend for step 3 : 1.9163954257965088 step4:brightness Tue May 6 11:43:25 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1746524488_576256_1356331946_84a33395ba90bcce8d40956792a4df03.jpg treat image : temp/1746524488_576256_1356331943_4c45e1c747017b9f5a174c85a57e7a86.jpg treat image : temp/1746524488_576256_1356331941_6cc4620dec2d438baefee5e9362e3c2f.jpg Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 3 time used for this insertion : 0.013781070709228516 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 3 time used for this insertion : 0.012136697769165039 save missing photos in datou_result : time spend for datou_step_exec : 4.9930524826049805 time spend to save output : 0.03101944923400879 total time spend for step 4 : 5.024071931838989 step5:rle_unique_nms_with_priority Tue May 6 11:43:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 162 chid ids of type : 4854 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 86 nb_hashtags : 8 time to prepare the origin masks : 87.79875159263611 time for calcul the mask position with numpy : 0.8352336883544922 nb_pixel_total : 4942084 time to create 1 rle with new method : 1.4328036308288574 time for calcul the mask position with numpy : 0.05331015586853027 nb_pixel_total : 111843 time to create 1 rle with old method : 0.12326407432556152 time for calcul the mask position with numpy : 0.05173015594482422 nb_pixel_total : 14188 time to create 1 rle with old method : 0.015091419219970703 time for calcul the mask position with numpy : 0.05127716064453125 nb_pixel_total : 39916 time to create 1 rle with old method : 0.043677330017089844 time for calcul the mask position with numpy : 0.05235743522644043 nb_pixel_total : 5826 time to create 1 rle with old method : 0.006559610366821289 time for calcul the mask position with numpy : 0.052129507064819336 nb_pixel_total : 5209 time to create 1 rle with old method : 0.0059051513671875 time for calcul the mask position with numpy : 0.05231976509094238 nb_pixel_total : 88383 time to create 1 rle with old method : 0.09697723388671875 time for calcul the mask position with numpy : 0.06177258491516113 nb_pixel_total : 336803 time to create 1 rle with new method : 1.3455140590667725 time for calcul the mask position with numpy : 0.060791730880737305 nb_pixel_total : 418969 time to create 1 rle with new method : 1.2073602676391602 time for calcul the mask position with numpy : 0.053197622299194336 nb_pixel_total : 6816 time to create 1 rle with old method : 0.008065223693847656 time for calcul the mask position with numpy : 0.05381202697753906 nb_pixel_total : 22569 time to create 1 rle with old method : 0.025067567825317383 time for calcul the mask position with numpy : 0.053945302963256836 nb_pixel_total : 20779 time to create 1 rle with old method : 0.023572206497192383 time for calcul the mask position with numpy : 0.05366802215576172 nb_pixel_total : 40305 time to create 1 rle with old method : 0.04771828651428223 time for calcul the mask position with numpy : 0.05661654472351074 nb_pixel_total : 57687 time to create 1 rle with old method : 0.06418633460998535 time for calcul the mask position with numpy : 0.05243086814880371 nb_pixel_total : 43180 time to create 1 rle with old method : 0.04802370071411133 time for calcul the mask position with numpy : 0.0530245304107666 nb_pixel_total : 113968 time to create 1 rle with old method : 0.12528204917907715 time for calcul the mask position with numpy : 0.05464935302734375 nb_pixel_total : 60720 time to create 1 rle with old method : 0.06593012809753418 time for calcul the mask position with numpy : 0.05275130271911621 nb_pixel_total : 17820 time to create 1 rle with old method : 0.020148277282714844 time for calcul the mask position with numpy : 0.053617000579833984 nb_pixel_total : 33237 time to create 1 rle with old method : 0.03909730911254883 time for calcul the mask position with numpy : 0.05406904220581055 nb_pixel_total : 107304 time to create 1 rle with old method : 0.11996579170227051 time for calcul the mask position with numpy : 0.05404329299926758 nb_pixel_total : 68112 time to create 1 rle with old method : 0.07692909240722656 time for calcul the mask position with numpy : 0.055072784423828125 nb_pixel_total : 7529 time to create 1 rle with old method : 0.008627891540527344 time for calcul the mask position with numpy : 0.05338788032531738 nb_pixel_total : 14287 time to create 1 rle with old method : 0.015921592712402344 time for calcul the mask position with numpy : 0.05367755889892578 nb_pixel_total : 13556 time to create 1 rle with old method : 0.015062332153320312 time for calcul the mask position with numpy : 0.053632259368896484 nb_pixel_total : 31839 time to create 1 rle with old method : 0.03653907775878906 time for calcul the mask position with numpy : 0.053734779357910156 nb_pixel_total : 2694 time to create 1 rle with old method : 0.003428220748901367 time for calcul the mask position with numpy : 0.05494117736816406 nb_pixel_total : 33301 time to create 1 rle with old method : 0.03740835189819336 time for calcul the mask position with numpy : 0.05393791198730469 nb_pixel_total : 62947 time to create 1 rle with old method : 0.07004165649414062 time for calcul the mask position with numpy : 0.05378246307373047 nb_pixel_total : 31742 time to create 1 rle with old method : 0.03540396690368652 time for calcul the mask position with numpy : 0.054605960845947266 nb_pixel_total : 71834 time to create 1 rle with old method : 0.08013629913330078 time for calcul the mask position with numpy : 0.05356287956237793 nb_pixel_total : 3288 time to create 1 rle with old method : 0.004074811935424805 time for calcul the mask position with numpy : 0.05473971366882324 nb_pixel_total : 154553 time to create 1 rle with new method : 1.5158321857452393 time for calcul the mask position with numpy : 0.05381917953491211 nb_pixel_total : 48374 time to create 1 rle with old method : 0.0537717342376709 time for calcul the mask position with numpy : 0.05383157730102539 nb_pixel_total : 54142 time to create 1 rle with old method : 0.06029224395751953 time for calcul the mask position with numpy : 0.05438351631164551 nb_pixel_total : 89585 time to create 1 rle with old method : 0.0995473861694336 time for calcul the mask position with numpy : 0.06120705604553223 nb_pixel_total : 227542 time to create 1 rle with new method : 1.1316158771514893 time for calcul the mask position with numpy : 0.051003456115722656 nb_pixel_total : 22457 time to create 1 rle with old method : 0.02661895751953125 time for calcul the mask position with numpy : 0.052661895751953125 nb_pixel_total : 89639 time to create 1 rle with old method : 0.09888815879821777 time for calcul the mask position with numpy : 0.05283951759338379 nb_pixel_total : 42670 time to create 1 rle with old method : 0.04697465896606445 time for calcul the mask position with numpy : 0.052716970443725586 nb_pixel_total : 43868 time to create 1 rle with old method : 0.04807925224304199 time for calcul the mask position with numpy : 0.05212664604187012 nb_pixel_total : 6752 time to create 1 rle with old method : 0.007806539535522461 time for calcul the mask position with numpy : 0.05232739448547363 nb_pixel_total : 1238 time to create 1 rle with old method : 0.001546621322631836 time for calcul the mask position with numpy : 0.054015159606933594 nb_pixel_total : 76308 time to create 1 rle with old method : 0.11980748176574707 time for calcul the mask position with numpy : 0.056475162506103516 nb_pixel_total : 424796 time to create 1 rle with new method : 1.0114092826843262 time for calcul the mask position with numpy : 0.05266094207763672 nb_pixel_total : 21606 time to create 1 rle with old method : 0.024487972259521484 time for calcul the mask position with numpy : 0.05367732048034668 nb_pixel_total : 36750 time to create 1 rle with old method : 0.04135274887084961 time for calcul the mask position with numpy : 0.059117794036865234 nb_pixel_total : 401941 time to create 1 rle with new method : 1.1466851234436035 time for calcul the mask position with numpy : 0.052846670150756836 nb_pixel_total : 246571 time to create 1 rle with new method : 0.9369752407073975 time for calcul the mask position with numpy : 0.05168581008911133 nb_pixel_total : 85347 time to create 1 rle with old method : 0.0909571647644043 time for calcul the mask position with numpy : 0.05221843719482422 nb_pixel_total : 37829 time to create 1 rle with old method : 0.042748451232910156 time for calcul the mask position with numpy : 0.0525975227355957 nb_pixel_total : 121603 time to create 1 rle with old method : 0.12735486030578613 time for calcul the mask position with numpy : 0.050333499908447266 nb_pixel_total : 64719 time to create 1 rle with old method : 0.06879973411560059 time for calcul the mask position with numpy : 0.05059313774108887 nb_pixel_total : 2233 time to create 1 rle with old method : 0.002621889114379883 time for calcul the mask position with numpy : 0.05086398124694824 nb_pixel_total : 37148 time to create 1 rle with old method : 0.039017438888549805 time for calcul the mask position with numpy : 0.050219058990478516 nb_pixel_total : 2256 time to create 1 rle with old method : 0.002390623092651367 time for calcul the mask position with numpy : 0.05066204071044922 nb_pixel_total : 38625 time to create 1 rle with old method : 0.042047977447509766 time for calcul the mask position with numpy : 0.052179813385009766 nb_pixel_total : 49570 time to create 1 rle with old method : 0.05424070358276367 time for calcul the mask position with numpy : 0.05488920211791992 nb_pixel_total : 339574 time to create 1 rle with new method : 1.0923032760620117 time for calcul the mask position with numpy : 0.050626516342163086 nb_pixel_total : 26801 time to create 1 rle with old method : 0.030658245086669922 time for calcul the mask position with numpy : 0.051187753677368164 nb_pixel_total : 131087 time to create 1 rle with old method : 0.13333368301391602 time for calcul the mask position with numpy : 0.04832649230957031 nb_pixel_total : 76680 time to create 1 rle with old method : 0.08018016815185547 time for calcul the mask position with numpy : 0.051975250244140625 nb_pixel_total : 45612 time to create 1 rle with old method : 0.05274796485900879 time for calcul the mask position with numpy : 0.05277204513549805 nb_pixel_total : 61856 time to create 1 rle with old method : 0.06882381439208984 time for calcul the mask position with numpy : 0.06135416030883789 nb_pixel_total : 666190 time to create 1 rle with new method : 0.9900953769683838 time for calcul the mask position with numpy : 0.053502559661865234 nb_pixel_total : 459076 time to create 1 rle with new method : 1.002779245376587 time for calcul the mask position with numpy : 0.05186724662780762 nb_pixel_total : 14039 time to create 1 rle with old method : 0.01720142364501953 time for calcul the mask position with numpy : 0.05382394790649414 nb_pixel_total : 70869 time to create 1 rle with old method : 0.08670544624328613 time for calcul the mask position with numpy : 0.05644798278808594 nb_pixel_total : 717903 time to create 1 rle with new method : 0.9913084506988525 time for calcul the mask position with numpy : 0.0509190559387207 nb_pixel_total : 85792 time to create 1 rle with old method : 0.0919501781463623 time for calcul the mask position with numpy : 0.05145573616027832 nb_pixel_total : 16313 time to create 1 rle with old method : 0.017308950424194336 time for calcul the mask position with numpy : 0.05117964744567871 nb_pixel_total : 32481 time to create 1 rle with old method : 0.03470730781555176 time for calcul the mask position with numpy : 0.05251336097717285 nb_pixel_total : 125113 time to create 1 rle with old method : 0.13332796096801758 time for calcul the mask position with numpy : 0.0511775016784668 nb_pixel_total : 139403 time to create 1 rle with old method : 0.1522233486175537 time for calcul the mask position with numpy : 0.05228304862976074 nb_pixel_total : 13747 time to create 1 rle with old method : 0.015271425247192383 time for calcul the mask position with numpy : 0.05217552185058594 nb_pixel_total : 53426 time to create 1 rle with old method : 0.05869650840759277 time for calcul the mask position with numpy : 0.0528712272644043 nb_pixel_total : 57875 time to create 1 rle with old method : 0.06202268600463867 time for calcul the mask position with numpy : 0.05083036422729492 nb_pixel_total : 4258 time to create 1 rle with old method : 0.0047686100006103516 time for calcul the mask position with numpy : 0.052693843841552734 nb_pixel_total : 83037 time to create 1 rle with old method : 0.0892934799194336 time for calcul the mask position with numpy : 0.05251145362854004 nb_pixel_total : 262426 time to create 1 rle with new method : 0.8102879524230957 time for calcul the mask position with numpy : 0.053739070892333984 nb_pixel_total : 65 time to create 1 rle with old method : 0.00012969970703125 time for calcul the mask position with numpy : 0.05517244338989258 nb_pixel_total : 4846 time to create 1 rle with old method : 0.005505800247192383 time for calcul the mask position with numpy : 0.05314016342163086 nb_pixel_total : 910 time to create 1 rle with old method : 0.0013337135314941406 time for calcul the mask position with numpy : 0.05316448211669922 nb_pixel_total : 82489 time to create 1 rle with old method : 0.0896751880645752 time for calcul the mask position with numpy : 0.05253338813781738 nb_pixel_total : 51013 time to create 1 rle with old method : 0.05501103401184082 time for calcul the mask position with numpy : 0.05120348930358887 nb_pixel_total : 61356 time to create 1 rle with old method : 0.06593704223632812 time for calcul the mask position with numpy : 0.052332401275634766 nb_pixel_total : 33628 time to create 1 rle with old method : 0.036641597747802734 time for calcul the mask position with numpy : 0.052562713623046875 nb_pixel_total : 4478 time to create 1 rle with old method : 0.005110502243041992 create new chi : 24.340279579162598 time to delete rle : 0.06061673164367676 batch 1 Loaded 87 chid ids of type : 4855 Number RLEs to save : 57265 TO DO : save crop sub photo not yet done ! save time : 3.6298654079437256 nb_obj : 69 nb_hashtags : 9 time to prepare the origin masks : 78.10934782028198 time for calcul the mask position with numpy : 0.596879243850708 nb_pixel_total : 6242120 time to create 1 rle with new method : 1.5966036319732666 time for calcul the mask position with numpy : 0.05540013313293457 nb_pixel_total : 58169 time to create 1 rle with old method : 0.06799530982971191 time for calcul the mask position with numpy : 0.058249711990356445 nb_pixel_total : 17675 time to create 1 rle with old method : 0.02855086326599121 time for calcul the mask position with numpy : 0.1685338020324707 nb_pixel_total : 329615 time to create 1 rle with new method : 1.0535645484924316 time for calcul the mask position with numpy : 0.05379056930541992 nb_pixel_total : 8504 time to create 1 rle with old method : 0.009737491607666016 time for calcul the mask position with numpy : 0.053882598876953125 nb_pixel_total : 35181 time to create 1 rle with old method : 0.03884387016296387 time for calcul the mask position with numpy : 0.05937361717224121 nb_pixel_total : 646682 time to create 1 rle with new method : 1.1194114685058594 time for calcul the mask position with numpy : 0.06411290168762207 nb_pixel_total : 113362 time to create 1 rle with old method : 0.12664437294006348 time for calcul the mask position with numpy : 0.05832099914550781 nb_pixel_total : 30888 time to create 1 rle with old method : 0.034548282623291016 time for calcul the mask position with numpy : 0.05357241630554199 nb_pixel_total : 33845 time to create 1 rle with old method : 0.03661227226257324 time for calcul the mask position with numpy : 0.05298352241516113 nb_pixel_total : 16060 time to create 1 rle with old method : 0.018160581588745117 time for calcul the mask position with numpy : 0.053426265716552734 nb_pixel_total : 76582 time to create 1 rle with old method : 0.08389735221862793 time for calcul the mask position with numpy : 0.0530545711517334 nb_pixel_total : 30622 time to create 1 rle with old method : 0.033928632736206055 time for calcul the mask position with numpy : 0.05436825752258301 nb_pixel_total : 14201 time to create 1 rle with old method : 0.02369546890258789 time for calcul the mask position with numpy : 0.05930662155151367 nb_pixel_total : 29518 time to create 1 rle with old method : 0.03195786476135254 time for calcul the mask position with numpy : 0.053116559982299805 nb_pixel_total : 19456 time to create 1 rle with old method : 0.021197080612182617 time for calcul the mask position with numpy : 0.05274057388305664 nb_pixel_total : 87297 time to create 1 rle with old method : 0.09996771812438965 time for calcul the mask position with numpy : 0.05412602424621582 nb_pixel_total : 79835 time to create 1 rle with old method : 0.1157994270324707 time for calcul the mask position with numpy : 0.05395197868347168 nb_pixel_total : 56287 time to create 1 rle with old method : 0.06367945671081543 time for calcul the mask position with numpy : 0.054251670837402344 nb_pixel_total : 268482 time to create 1 rle with new method : 1.436523675918579 time for calcul the mask position with numpy : 0.05952906608581543 nb_pixel_total : 294465 time to create 1 rle with new method : 1.2947235107421875 time for calcul the mask position with numpy : 0.05736684799194336 nb_pixel_total : 56671 time to create 1 rle with old method : 0.06308650970458984 time for calcul the mask position with numpy : 0.05356597900390625 nb_pixel_total : 114583 time to create 1 rle with old method : 0.1316990852355957 time for calcul the mask position with numpy : 0.051371097564697266 nb_pixel_total : 34504 time to create 1 rle with old method : 0.04076099395751953 time for calcul the mask position with numpy : 0.05188870429992676 nb_pixel_total : 30720 time to create 1 rle with old method : 0.03663158416748047 time for calcul the mask position with numpy : 0.06072807312011719 nb_pixel_total : 1053 time to create 1 rle with old method : 0.002388477325439453 time for calcul the mask position with numpy : 0.05674910545349121 nb_pixel_total : 259367 time to create 1 rle with new method : 1.3839054107666016 time for calcul the mask position with numpy : 0.052934885025024414 nb_pixel_total : 34011 time to create 1 rle with old method : 0.03675580024719238 time for calcul the mask position with numpy : 0.0527348518371582 nb_pixel_total : 43337 time to create 1 rle with old method : 0.04704022407531738 time for calcul the mask position with numpy : 0.05222511291503906 nb_pixel_total : 9306 time to create 1 rle with old method : 0.010570049285888672 time for calcul the mask position with numpy : 0.05330991744995117 nb_pixel_total : 15011 time to create 1 rle with old method : 0.01635599136352539 time for calcul the mask position with numpy : 0.05401134490966797 nb_pixel_total : 15849 time to create 1 rle with old method : 0.025885820388793945 time for calcul the mask position with numpy : 0.05951094627380371 nb_pixel_total : 53320 time to create 1 rle with old method : 0.05937457084655762 time for calcul the mask position with numpy : 0.05322527885437012 nb_pixel_total : 40013 time to create 1 rle with old method : 0.04430365562438965 time for calcul the mask position with numpy : 0.05633234977722168 nb_pixel_total : 1676 time to create 1 rle with old method : 0.0019638538360595703 time for calcul the mask position with numpy : 0.053106069564819336 nb_pixel_total : 93952 time to create 1 rle with old method : 0.10222601890563965 time for calcul the mask position with numpy : 0.05275440216064453 nb_pixel_total : 76322 time to create 1 rle with old method : 0.08313870429992676 time for calcul the mask position with numpy : 0.0542752742767334 nb_pixel_total : 448538 time to create 1 rle with new method : 1.1171245574951172 time for calcul the mask position with numpy : 0.056011199951171875 nb_pixel_total : 339254 time to create 1 rle with new method : 0.9957988262176514 time for calcul the mask position with numpy : 0.05151963233947754 nb_pixel_total : 36295 time to create 1 rle with old method : 0.04012918472290039 time for calcul the mask position with numpy : 0.05276370048522949 nb_pixel_total : 122539 time to create 1 rle with old method : 0.13203954696655273 time for calcul the mask position with numpy : 0.05606484413146973 nb_pixel_total : 255690 time to create 1 rle with new method : 1.0787005424499512 time for calcul the mask position with numpy : 0.061882734298706055 nb_pixel_total : 123740 time to create 1 rle with old method : 0.1358339786529541 time for calcul the mask position with numpy : 0.05421018600463867 nb_pixel_total : 52914 time to create 1 rle with old method : 0.060584306716918945 time for calcul the mask position with numpy : 0.05381655693054199 nb_pixel_total : 75316 time to create 1 rle with old method : 0.08467793464660645 time for calcul the mask position with numpy : 0.052114248275756836 nb_pixel_total : 15704 time to create 1 rle with old method : 0.018067121505737305 time for calcul the mask position with numpy : 0.05672454833984375 nb_pixel_total : 175150 time to create 1 rle with new method : 1.081040382385254 time for calcul the mask position with numpy : 0.05537915229797363 nb_pixel_total : 187815 time to create 1 rle with new method : 1.000114917755127 time for calcul the mask position with numpy : 0.05008125305175781 nb_pixel_total : 12083 time to create 1 rle with old method : 0.012706995010375977 time for calcul the mask position with numpy : 0.05162405967712402 nb_pixel_total : 85529 time to create 1 rle with old method : 0.09061074256896973 time for calcul the mask position with numpy : 0.05203437805175781 nb_pixel_total : 105589 time to create 1 rle with old method : 0.11303520202636719 time for calcul the mask position with numpy : 0.051395416259765625 nb_pixel_total : 85699 time to create 1 rle with old method : 0.09298157691955566 time for calcul the mask position with numpy : 0.051965951919555664 nb_pixel_total : 32168 time to create 1 rle with old method : 0.033483028411865234 time for calcul the mask position with numpy : 0.05307722091674805 nb_pixel_total : 98180 time to create 1 rle with old method : 0.10779929161071777 time for calcul the mask position with numpy : 0.05402112007141113 nb_pixel_total : 58845 time to create 1 rle with old method : 0.064697265625 time for calcul the mask position with numpy : 0.05286884307861328 nb_pixel_total : 112290 time to create 1 rle with old method : 0.12160611152648926 time for calcul the mask position with numpy : 0.053354501724243164 nb_pixel_total : 160527 time to create 1 rle with new method : 1.025639295578003 time for calcul the mask position with numpy : 0.05202364921569824 nb_pixel_total : 38655 time to create 1 rle with old method : 0.04264330863952637 time for calcul the mask position with numpy : 0.05054473876953125 nb_pixel_total : 27265 time to create 1 rle with old method : 0.028894901275634766 time for calcul the mask position with numpy : 0.05289745330810547 nb_pixel_total : 412503 time to create 1 rle with new method : 1.0517513751983643 time for calcul the mask position with numpy : 0.052314043045043945 nb_pixel_total : 66619 time to create 1 rle with old method : 0.07597494125366211 time for calcul the mask position with numpy : 0.05466961860656738 nb_pixel_total : 160804 time to create 1 rle with new method : 1.1199946403503418 time for calcul the mask position with numpy : 0.050977468490600586 nb_pixel_total : 10106 time to create 1 rle with old method : 0.011339902877807617 time for calcul the mask position with numpy : 0.05227518081665039 nb_pixel_total : 15950 time to create 1 rle with old method : 0.01763176918029785 time for calcul the mask position with numpy : 0.05304241180419922 nb_pixel_total : 23511 time to create 1 rle with old method : 0.02634453773498535 time for calcul the mask position with numpy : 0.053766727447509766 nb_pixel_total : 87136 time to create 1 rle with old method : 0.09399819374084473 time for calcul the mask position with numpy : 0.0529935359954834 nb_pixel_total : 122269 time to create 1 rle with old method : 0.12943673133850098 time for calcul the mask position with numpy : 0.05164909362792969 nb_pixel_total : 29754 time to create 1 rle with old method : 0.032416343688964844 time for calcul the mask position with numpy : 0.05170011520385742 nb_pixel_total : 16997 time to create 1 rle with old method : 0.018507957458496094 time for calcul the mask position with numpy : 0.051474571228027344 nb_pixel_total : 15225 time to create 1 rle with old method : 0.01649785041809082 create new chi : 24.554712057113647 time to delete rle : 0.008151531219482422 batch 1 Loaded 70 chid ids of type : 4855 Number RLEs to save : 54492 TO DO : save crop sub photo not yet done ! save time : 2.9863388538360596 nb_obj : 6 nb_hashtags : 4 time to prepare the origin masks : 8.676207304000854 time for calcul the mask position with numpy : 0.9991679191589355 nb_pixel_total : 8533962 time to create 1 rle with new method : 2.139671564102173 time for calcul the mask position with numpy : 0.040229082107543945 nb_pixel_total : 31955 time to create 1 rle with old method : 0.037540435791015625 time for calcul the mask position with numpy : 0.041938066482543945 nb_pixel_total : 175448 time to create 1 rle with new method : 1.0277163982391357 time for calcul the mask position with numpy : 0.04449057579040527 nb_pixel_total : 166464 time to create 1 rle with new method : 1.6728506088256836 time for calcul the mask position with numpy : 0.10277485847473145 nb_pixel_total : 1536762 time to create 1 rle with new method : 1.2596027851104736 time for calcul the mask position with numpy : 0.07062840461730957 nb_pixel_total : 1131387 time to create 1 rle with new method : 2.0411174297332764 time for calcul the mask position with numpy : 0.0854039192199707 nb_pixel_total : 1403222 time to create 1 rle with new method : 0.9630439281463623 create new chi : 10.828712701797485 time to delete rle : 0.0023984909057617188 batch 1 Loaded 7 chid ids of type : 4855 Number RLEs to save : 19642 TO DO : save crop sub photo not yet done ! save time : 1.149876594543457 map_output_result : {1356331946: (0.620587784035315, 'Should be the crop_list due to order', 0.45728598487363875), 1356331943: (0.620587784035315, 'Should be the crop_list due to order', 0.49308454442217536), 1356331941: (0.620587784035315, 'Should be the crop_list due to order', 0.911392822810131)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1356331946, 1356331943, 1356331941] Looping around the photos to save general results len do output : 3 /1356331946.Didn't retrieve data . /1356331943.Didn't retrieve data . /1356331941.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2797324') ('4741', '22713351', '1356331946', None, None, None, None, None, '2797324') ('4741', None, None, None, None, None, None, None, '2797324') ('4741', '22713351', '1356331943', None, None, None, None, None, '2797324') ('4741', None, None, None, None, None, None, None, '2797324') ('4741', '22713351', '1356331941', None, None, None, None, None, '2797324') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.012719869613647461 save_final save missing photos in datou_result : time spend for datou_step_exec : 243.79965353012085 time spend to save output : 0.017149925231933594 total time spend for step 5 : 243.81680345535278 step6:crop_condition Tue May 6 11:47:34 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 4855 Loading chi in step crop for list_pids : 3 ! batch 1 Loaded 164 chid ids of type : 4855 begin to crop the class : papier param for this class : {'min_score': 0.1} filtre for class : papier hashtag_id of this class : 492668766 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 11 About to insert : list_path_to_insert length 11 new photo from crops ! About to upload 11 photos upload in portfolio : 3015255 init cache_photo without model_param we have 11 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746524863_576256 we have uploaded 11 photos in the portfolio 3015255 time of upload the photos Elapsed time : 3.3930957317352295 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.1} filtre for class : carton hashtag_id of this class : 492774966 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 47 About to insert : list_path_to_insert length 47 new photo from crops ! About to upload 47 photos upload in portfolio : 3015255 init cache_photo without model_param we have 47 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746524889_576256 we have uploaded 47 photos in the portfolio 3015255 time of upload the photos Elapsed time : 16.35191035270691 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.1} filtre for class : metal hashtag_id of this class : 492628673 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 15 About to insert : list_path_to_insert length 15 new photo from crops ! About to upload 15 photos upload in portfolio : 3015255 init cache_photo without model_param we have 15 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746524911_576256 we have uploaded 15 photos in the portfolio 3015255 time of upload the photos Elapsed time : 3.8299617767333984 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.1} filtre for class : pet_clair hashtag_id of this class : 2107755846 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3015255 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746524917_576256 we have uploaded 2 photos in the portfolio 3015255 time of upload the photos Elapsed time : 0.8385715484619141 we have finished the crop for the class : pet_clair begin to crop the class : pehd param for this class : {'min_score': 0.1} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.1} filtre for class : pet_fonce hashtag_id of this class : 2107755900 begin to crop the class : sac param for this class : {'min_score': 0.1} filtre for class : sac hashtag_id of this class : 492618350 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 15 About to insert : list_path_to_insert length 15 new photo from crops ! About to upload 15 photos upload in portfolio : 3015255 init cache_photo without model_param we have 15 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746524927_576256 we have uploaded 15 photos in the portfolio 3015255 time of upload the photos Elapsed time : 6.819588661193848 we have finished the crop for the class : sac begin to crop the class : textiles param for this class : {'min_score': 0.1} filtre for class : textiles hashtag_id of this class : 494789423 map_result returned by crop_photo_return_map_crop : length : 0 About to insert : list_path_to_insert length 0 new photo from crops ! About to upload 0 photos WARNING : list_path_to_insert is empty, cannot upload ! we have finished the crop for the class : textiles begin to crop the class : verre param for this class : {'min_score': 0.1} filtre for class : verre hashtag_id of this class : 492631919 begin to crop the class : organique param for this class : {'min_score': 0.1} filtre for class : organique hashtag_id of this class : 530726390 begin to crop the class : dasri param for this class : {'min_score': 0.1} filtre for class : dasri hashtag_id of this class : 1608424396 begin to crop the class : masque param for this class : {'min_score': 0.1} filtre for class : masque hashtag_id of this class : 492626781 begin to crop the class : encombrant param for this class : {'min_score': 0.1} filtre for class : encombrant hashtag_id of this class : 605975549 begin to crop the class : autre param for this class : {'min_score': 0.1} filtre for class : autre hashtag_id of this class : 494826614 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 15 About to insert : list_path_to_insert length 15 new photo from crops ! About to upload 15 photos upload in portfolio : 3015255 init cache_photo without model_param we have 15 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746524959_576256 we have uploaded 15 photos in the portfolio 3015255 time of upload the photos Elapsed time : 7.947225093841553 we have finished the crop for the class : autre begin to crop the class : autre_emballage param for this class : {'min_score': 0.1} filtre for class : autre_emballage hashtag_id of this class : 2107760328 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3015255 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746524969_576256 we have uploaded 1 photos in the portfolio 3015255 time of upload the photos Elapsed time : 0.6587514877319336 we have finished the crop for the class : autre_emballage begin to crop the class : autre_non_emballage param for this class : {'min_score': 0.1} filtre for class : autre_non_emballage hashtag_id of this class : 2107760329 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 3015255 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746524972_576256 we have uploaded 3 photos in the portfolio 3015255 time of upload the photos Elapsed time : 1.21702241897583 we have finished the crop for the class : autre_non_emballage begin to crop the class : environnement param for this class : {'min_score': 0.1} filtre for class : environnement hashtag_id of this class : 493012381 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 21 About to insert : list_path_to_insert length 21 new photo from crops ! About to upload 21 photos upload in portfolio : 3015255 init cache_photo without model_param we have 21 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746525078_576256 we have uploaded 21 photos in the portfolio 3015255 time of upload the photos Elapsed time : 7.824714422225952 we have finished the crop for the class : environnement delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1356331946, 1356331943, 1356331941] Looping around the photos to save general results len do output : 130 /1356333477Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333479Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333480Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333481Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333482Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333483Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333484Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333486Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333487Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333488Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333489Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333509Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333510Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333511Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333512Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333513Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333514Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333515Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333516Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333517Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333518Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333519Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333520Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333521Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333522Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333523Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333524Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333525Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333526Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333527Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333528Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333529Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333530Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333531Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333532Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333533Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333534Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333535Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333536Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333537Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333538Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333539Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333540Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333541Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333542Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333543Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333544Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333545Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333546Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333547Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333548Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333549Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333550Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333551Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333552Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333553Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333554Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333555Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333560Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333561Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333562Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333563Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333564Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333565Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333566Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333567Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333568Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333569Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333570Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333571Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333572Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333573Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333574Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333577Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333578Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333593Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333594Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333595Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333596Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333597Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333598Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333599Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333600Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333601Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333602Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333603Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333604Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333605Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333606Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333607Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333620Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333621Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333622Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333623Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333624Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333625Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333626Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333627Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333628Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333629Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333630Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333631Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333632Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333634Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333635Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333637Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333639Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333640Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333641Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333737Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333738Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333739Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333740Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333741Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333742Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333743Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333744Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333745Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333746Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333747Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333748Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333749Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333750Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333751Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333752Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333753Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333754Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333755Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333756Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356333757Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2797324') ('4741', '22713351', '1356331946', None, None, None, None, None, '2797324') ('4741', None, None, None, None, None, None, None, '2797324') ('4741', '22713351', '1356331943', None, None, None, None, None, '2797324') ('4741', None, None, None, None, None, None, None, '2797324') ('4741', '22713351', '1356331941', None, None, None, None, None, '2797324') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 393 time used for this insertion : 0.042366981506347656 save_final save missing photos in datou_result : time spend for datou_step_exec : 232.3411135673523 time spend to save output : 0.048929452896118164 total time spend for step 6 : 232.3900430202484 step7:ventilate_hashtags_in_portfolio Tue May 6 11:51:26 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 22713351 get user id for portfolio 22713351 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22713351 AND mptpi.`type`=4855 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22713351 AND mptpi.`type`=4855 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22713351 AND mptpi.`type`=4856 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22713351 AND mptpi.`type`=4856 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do lien utilise dans velours : https://www.fotonower.com/velours/22713585,22713586,22713587,22713588,22713589,22713594,22713596,22713597,22713598,22713599,22713600,22713601,22713603,22713604,22713605,22713606,22713607,22713608,22713609,22713610?tags=papier,carton,metal,pet_clair,pehd,ela,textiles,verre,organique,dasri,masque,encombrant,autre_non_emballage,environnement,autre,flux_dev,mal_croppe,flou,film_dev_souple,sac_om_plein&datou_id_consolidate=4742&port_consolidate=22713351 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1356331946, 1356331943, 1356331941] Looping around the photos to save general results len do output : 1 /22713351. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2797324') ('4741', '22713351', '1356331946', None, None, None, None, None, '2797324') ('4741', None, None, None, None, None, None, None, '2797324') ('4741', '22713351', '1356331943', None, None, None, None, None, '2797324') ('4741', None, None, None, None, None, None, None, '2797324') ('4741', '22713351', '1356331941', None, None, None, None, None, '2797324') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 4 time used for this insertion : 0.013652563095092773 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.105973720550537 time spend to save output : 0.013901710510253906 total time spend for step 7 : 4.119875431060791 step8:final Tue May 6 11:51:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1356331946: ('0.525598599831018',), 1356331943: ('0.525598599831018',), 1356331941: ('0.525598599831018',)} new output for save of step final : {1356331946: ('0.525598599831018',), 1356331943: ('0.525598599831018',), 1356331941: ('0.525598599831018',)} [1356331946, 1356331943, 1356331941] Looping around the photos to save general results len do output : 3 /1356331946.Didn't retrieve data . /1356331943.Didn't retrieve data . /1356331941.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2797324') ('4741', '22713351', '1356331946', None, None, None, None, None, '2797324') ('4741', None, None, None, None, None, None, None, '2797324') ('4741', '22713351', '1356331943', None, None, None, None, None, '2797324') ('4741', None, None, None, None, None, None, None, '2797324') ('4741', '22713351', '1356331941', None, None, None, None, None, '2797324') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.012786865234375 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.11388421058654785 time spend to save output : 0.013196706771850586 total time spend for step 8 : 0.12708091735839844 step9:velours_tree Tue May 6 11:51:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 1.377427339553833 time spend to save output : 5.602836608886719e-05 total time spend for step 9 : 1.3774833679199219 step10:send_mail_cod Tue May 6 11:51:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P22713351_06-05-2025_11_51_32.pdf 22713559 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227135591746525092 22713560 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227135601746525093 22713561 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227135611746525094 22713562 change filename to text .change filename to text .imagette227135621746525095 22713563 imagette227135631746525096 22713564 imagette227135641746525096 22713565 imagette227135651746525096 22713566 imagette227135661746525096 22713567 imagette227135671746525096 22713568 imagette227135681746525096 22713569 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227135691746525096 22713570 imagette227135701746525097 22713571 imagette227135711746525097 22713572 imagette227135721746525097 22713573 imagette227135731746525097 22713574 imagette227135741746525097 22713575 imagette227135751746525097 22713576 change filename to text .imagette227135761746525097 22713577 change filename to text .change filename to text .change filename to text .imagette227135771746525097 22713579 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227135791746525097 22713580 imagette227135801746525099 22713581 imagette227135811746525099 22713582 imagette227135821746525099 22713583 imagette227135831746525099 22713584 imagette227135841746525099 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=22713351 and hashtag_type = 4855 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/22713585,22713586,22713587,22713588,22713589,22713594,22713596,22713597,22713598,22713599,22713600,22713601,22713603,22713604,22713605,22713606,22713607,22713608,22713609,22713610?tags=papier,carton,metal,pet_clair,pehd,ela,textiles,verre,organique,dasri,masque,encombrant,autre_non_emballage,environnement,autre,flux_dev,mal_croppe,flou,film_dev_souple,sac_om_plein&datou_id_consolidate=4742&port_consolidate=22713351 your option no_mail is active, we will not send the real mail to your client args[1356331946] : ((1356331946, 3216.375394352539, 492609224), (1356331946, -0.22688079377520343, 496442774), '0.525598599831018') We are sending mail with results at report@fotonower.com,cod@fotonower.com args[1356331943] : ((1356331943, 4146.6991381408625, 492609224), (1356331943, -0.3107514634279765, 496442774), '0.525598599831018') We are sending mail with results at report@fotonower.com,cod@fotonower.com args[1356331941] : ((1356331941, 180.04659657686707, 492688767), (1356331941, 0.042468400882512, 2107752395), '0.525598599831018') We are sending mail with results at report@fotonower.com,cod@fotonower.com refus_total : 0.525598599831018 2022-04-13 10:29:59 0 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22713351_06-05-2025_11_51_32.pdf results_Auto_P22713351_06-05-2025_11_51_32.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22713351_06-05-2025_11_51_32.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4741','22713351','results_Auto_P22713351_06-05-2025_11_51_32.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22713351_06-05-2025_11_51_32.pdf','pdf','','1.82','0.525598599831018') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1356331946, 1356331943, 1356331941] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2797324') ('4741', '22713351', '1356331946', None, None, None, None, None, '2797324') ('4741', None, None, None, None, None, None, None, '2797324') ('4741', '22713351', '1356331943', None, None, None, None, None, '2797324') ('4741', None, None, None, None, None, None, None, '2797324') ('4741', '22713351', '1356331941', None, None, None, None, None, '2797324') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.012366056442260742 save_final save missing photos in datou_result : time spend for datou_step_exec : 9.39325475692749 time spend to save output : 0.012675046920776367 total time spend for step 10 : 9.405929803848267 step11:split_time_score Tue May 6 11:51:41 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('04', 2), ('05', 1)) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 03052025 22713351 Nombre de photos uploadées : 3 / 23040 (0%) 03052025 22713351 Nombre de photos taguées (types de déchets): 0 / 3 (0%) 03052025 22713351 Nombre de photos taguées (volume) : 0 / 3 (0%) elapsed_time : load_data_split_time_score 3.337860107421875e-06 elapsed_time : order_list_meta_photo_and_scores 1.1444091796875e-05 ??? elapsed_time : fill_and_build_computed_from_old_data 0.0003311634063720703 elapsed_time : insert_dashboard_record_day_entry 0.023519039154052734 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.525598599831018 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22713351_06-05-2025_11_51_32.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22713351 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22713351 AND mptpi.`type`=4855 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'03052025': {'nb_upload': 3, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1356331946, 1356331943, 1356331941] Looping around the photos to save general results len do output : 1 /22713351Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2797324') ('4741', '22713351', '1356331946', None, None, None, None, None, '2797324') ('4741', None, None, None, None, None, None, None, '2797324') ('4741', '22713351', '1356331943', None, None, None, None, None, '2797324') ('4741', None, None, None, None, None, None, None, '2797324') ('4741', '22713351', '1356331941', None, None, None, None, None, '2797324') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 4 time used for this insertion : 0.013734102249145508 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.24816274642944336 time spend to save output : 0.013985157012939453 total time spend for step 11 : 0.2621479034423828 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 1 len(input) + len(total_photo_id_missing) : 3 set_done_treatment 356.20user 477.79system 10:17.51elapsed 135%CPU (0avgtext+0avgdata 5922524maxresident)k 1410784inputs+445640outputs (18605major+31482018minor)pagefaults 0swaps