python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 437681 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['3941696'] with mtr_portfolio_ids : ['27805408'] and first list_photo_ids : [] new path : /proc/437681/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 15 ; length of list_pids : 15 ; length of list_args : 15 time to download the photos : 2.330861806869507 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Tue Oct 14 17:50:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10998 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-10-14 17:50:33.117641: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-10-14 17:50:33.142518: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493010000 Hz 2025-10-14 17:50:33.144542: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f11e8000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-10-14 17:50:33.144589: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-10-14 17:50:33.148216: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-10-14 17:50:33.466208: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x30ecda30 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-10-14 17:50:33.466260: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-10-14 17:50:33.467717: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-10-14 17:50:33.471433: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-10-14 17:50:33.474619: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-10-14 17:50:33.477184: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-10-14 17:50:33.477676: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-10-14 17:50:33.480512: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-10-14 17:50:33.481771: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-10-14 17:50:33.486812: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-10-14 17:50:33.488559: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-10-14 17:50:33.488624: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-10-14 17:50:33.489550: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-10-14 17:50:33.489568: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-10-14 17:50:33.489578: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-10-14 17:50:33.491306: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10193 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-10-14 17:50:33.877909: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-10-14 17:50:33.878027: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-10-14 17:50:33.878064: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-10-14 17:50:33.878096: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-10-14 17:50:33.878125: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-10-14 17:50:33.878156: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-10-14 17:50:33.878186: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-10-14 17:50:33.878218: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-10-14 17:50:33.880241: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-10-14 17:50:33.882098: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-10-14 17:50:33.882176: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-10-14 17:50:33.882211: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-10-14 17:50:33.882241: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-10-14 17:50:33.882271: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-10-14 17:50:33.882338: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-10-14 17:50:33.882365: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-10-14 17:50:33.882391: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-10-14 17:50:33.883891: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-10-14 17:50:33.883928: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-10-14 17:50:33.883937: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-10-14 17:50:33.883945: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-10-14 17:50:33.885300: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10193 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-10-14 17:50:40.700030: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-10-14 17:50:40.859624: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 15 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 32.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 27.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 28.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 14 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 11.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 7.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 15 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 23.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 34.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 38.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 43.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 32.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 72.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 31.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 22.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 40.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 46.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 Detection mask done ! Trying to reset tf kernel 438232 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5706 tf kernel not reseted sub process len(results) : 15 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 15 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10998 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.00033164024353027344 nb_pixel_total : 16378 time to create 1 rle with old method : 0.018402814865112305 length of segment : 130 time for calcul the mask position with numpy : 0.0001456737518310547 nb_pixel_total : 7105 time to create 1 rle with old method : 0.007831573486328125 length of segment : 151 time for calcul the mask position with numpy : 0.00010037422180175781 nb_pixel_total : 4812 time to create 1 rle with old method : 0.0055735111236572266 length of segment : 69 time for calcul the mask position with numpy : 6.0558319091796875e-05 nb_pixel_total : 2764 time to create 1 rle with old method : 0.0031974315643310547 length of segment : 47 time for calcul the mask position with numpy : 0.0012063980102539062 nb_pixel_total : 79188 time to create 1 rle with old method : 0.08571481704711914 length of segment : 239 time for calcul the mask position with numpy : 0.00012993812561035156 nb_pixel_total : 6444 time to create 1 rle with old method : 0.007528543472290039 length of segment : 83 time for calcul the mask position with numpy : 7.748603820800781e-05 nb_pixel_total : 2445 time to create 1 rle with old method : 0.002846240997314453 length of segment : 101 time for calcul the mask position with numpy : 7.605552673339844e-05 nb_pixel_total : 3997 time to create 1 rle with old method : 0.004853248596191406 length of segment : 58 time for calcul the mask position with numpy : 0.00010156631469726562 nb_pixel_total : 5781 time to create 1 rle with old method : 0.006487846374511719 length of segment : 78 time for calcul the mask position with numpy : 0.00010371208190917969 nb_pixel_total : 6436 time to create 1 rle with old method : 0.007245540618896484 length of segment : 106 time for calcul the mask position with numpy : 5.602836608886719e-05 nb_pixel_total : 2239 time to create 1 rle with old method : 0.0025730133056640625 length of segment : 51 time for calcul the mask position with numpy : 0.00015211105346679688 nb_pixel_total : 10613 time to create 1 rle with old method : 0.011822223663330078 length of segment : 130 time for calcul the mask position with numpy : 0.00014162063598632812 nb_pixel_total : 6971 time to create 1 rle with old method : 0.008325815200805664 length of segment : 105 time for calcul the mask position with numpy : 0.00015592575073242188 nb_pixel_total : 4267 time to create 1 rle with old method : 0.005162239074707031 length of segment : 70 time for calcul the mask position with numpy : 0.0005524158477783203 nb_pixel_total : 38949 time to create 1 rle with old method : 0.04792356491088867 length of segment : 184 time for calcul the mask position with numpy : 8.606910705566406e-05 nb_pixel_total : 3531 time to create 1 rle with old method : 0.004113435745239258 length of segment : 69 time for calcul the mask position with numpy : 0.00014281272888183594 nb_pixel_total : 8686 time to create 1 rle with old method : 0.009606361389160156 length of segment : 114 time for calcul the mask position with numpy : 0.00011324882507324219 nb_pixel_total : 6723 time to create 1 rle with old method : 0.007486820220947266 length of segment : 123 time for calcul the mask position with numpy : 6.723403930664062e-05 nb_pixel_total : 3439 time to create 1 rle with old method : 0.003934621810913086 length of segment : 63 time for calcul the mask position with numpy : 0.00011563301086425781 nb_pixel_total : 5943 time to create 1 rle with old method : 0.006651878356933594 length of segment : 154 time for calcul the mask position with numpy : 7.605552673339844e-05 nb_pixel_total : 2513 time to create 1 rle with old method : 0.003106355667114258 length of segment : 54 time for calcul the mask position with numpy : 0.0001595020294189453 nb_pixel_total : 10262 time to create 1 rle with old method : 0.011897802352905273 length of segment : 124 time for calcul the mask position with numpy : 5.936622619628906e-05 nb_pixel_total : 1576 time to create 1 rle with old method : 0.0018525123596191406 length of segment : 38 time for calcul the mask position with numpy : 0.0002911090850830078 nb_pixel_total : 22839 time to create 1 rle with old method : 0.026140928268432617 length of segment : 209 time for calcul the mask position with numpy : 0.00020122528076171875 nb_pixel_total : 10895 time to create 1 rle with old method : 0.012056350708007812 length of segment : 217 time for calcul the mask position with numpy : 0.000164031982421875 nb_pixel_total : 10413 time to create 1 rle with old method : 0.011520862579345703 length of segment : 140 time for calcul the mask position with numpy : 0.00014710426330566406 nb_pixel_total : 10811 time to create 1 rle with old method : 0.011816740036010742 length of segment : 133 time for calcul the mask position with numpy : 4.982948303222656e-05 nb_pixel_total : 1779 time to create 1 rle with old method : 0.002169370651245117 length of segment : 46 time for calcul the mask position with numpy : 0.00023245811462402344 nb_pixel_total : 13996 time to create 1 rle with old method : 0.019884109497070312 length of segment : 140 time for calcul the mask position with numpy : 0.009542226791381836 nb_pixel_total : 647531 time to create 1 rle with new method : 0.023071765899658203 length of segment : 1355 time for calcul the mask position with numpy : 0.00020766258239746094 nb_pixel_total : 11873 time to create 1 rle with old method : 0.01335906982421875 length of segment : 122 time for calcul the mask position with numpy : 0.00011730194091796875 nb_pixel_total : 6073 time to create 1 rle with old method : 0.006973743438720703 length of segment : 107 time for calcul the mask position with numpy : 0.0001266002655029297 nb_pixel_total : 7136 time to create 1 rle with old method : 0.008024215698242188 length of segment : 144 time for calcul the mask position with numpy : 0.0001666545867919922 nb_pixel_total : 11082 time to create 1 rle with old method : 0.012537240982055664 length of segment : 144 time for calcul the mask position with numpy : 6.937980651855469e-05 nb_pixel_total : 2324 time to create 1 rle with old method : 0.002674102783203125 length of segment : 78 time for calcul the mask position with numpy : 0.00011754035949707031 nb_pixel_total : 5740 time to create 1 rle with old method : 0.006470441818237305 length of segment : 86 time for calcul the mask position with numpy : 0.00022983551025390625 nb_pixel_total : 16790 time to create 1 rle with old method : 0.01828455924987793 length of segment : 167 time for calcul the mask position with numpy : 9.703636169433594e-05 nb_pixel_total : 6310 time to create 1 rle with old method : 0.007333278656005859 length of segment : 66 time for calcul the mask position with numpy : 6.556510925292969e-05 nb_pixel_total : 3068 time to create 1 rle with old method : 0.003679513931274414 length of segment : 56 time for calcul the mask position with numpy : 0.0001964569091796875 nb_pixel_total : 12562 time to create 1 rle with old method : 0.014106988906860352 length of segment : 135 time for calcul the mask position with numpy : 0.0001595020294189453 nb_pixel_total : 6165 time to create 1 rle with old method : 0.007392168045043945 length of segment : 133 time spent for convertir_results : 1.8798425197601318 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 41 chid ids of type : 3594 Number RLEs to save : 5819 save missing photos in datou_result : time spend for datou_step_exec : 22.51044464111328 time spend to save output : 0.8236463069915771 total time spend for step 1 : 23.33409094810486 step2:crop_condition Tue Oct 14 17:50:53 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 15 ! batch 1 Loaded 41 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 32 About to insert : list_path_to_insert length 32 new photo from crops ! About to upload 32 photos upload in portfolio : 3736932 init cache_photo without model_param we have 32 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1760457056_437681 INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435458_1711d6fb738f9979c690a0002136a339_rle_crop_3997860639_0.png', 0, 176, 130, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435458_1711d6fb738f9979c690a0002136a339_rle_crop_3997860640_0.png', 0, 85, 151, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435456_d5694014c6fe3c5607f85e1155e2ff51_rle_crop_3997860643_0.png', 0, 552, 212, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435456_d5694014c6fe3c5607f85e1155e2ff51_rle_crop_3997860642_0.png', 0, 70, 46, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435456_d5694014c6fe3c5607f85e1155e2ff51_rle_crop_3997860641_0.png', 0, 99, 69, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435454_ca070e1d964316b528f1bed6e2d2a0ee_rle_crop_3997860646_0.png', 0, 116, 47, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435454_ca070e1d964316b528f1bed6e2d2a0ee_rle_crop_3997860647_0.png', 0, 104, 78, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435454_ca070e1d964316b528f1bed6e2d2a0ee_rle_crop_3997860644_0.png', 0, 114, 83, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435453_748631fd1d9c7b538d7a8aeadbdeba0b_rle_crop_3997860651_0.png', 0, 123, 103, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435453_748631fd1d9c7b538d7a8aeadbdeba0b_rle_crop_3997860649_0.png', 0, 56, 51, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435453_748631fd1d9c7b538d7a8aeadbdeba0b_rle_crop_3997860650_0.png', 0, 116, 130, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435451_73d2f19f8212a6892ca6e54b07610b48_rle_crop_3997860653_0.png', 0, 340, 155, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435451_73d2f19f8212a6892ca6e54b07610b48_rle_crop_3997860654_0.png', 0, 94, 68, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435451_73d2f19f8212a6892ca6e54b07610b48_rle_crop_3997860656_0.png', 0, 74, 122, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435451_73d2f19f8212a6892ca6e54b07610b48_rle_crop_3997860657_0.png', 0, 72, 61, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435451_73d2f19f8212a6892ca6e54b07610b48_rle_crop_3997860652_0.png', 0, 78, 70, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435451_73d2f19f8212a6892ca6e54b07610b48_rle_crop_3997860655_0.png', 0, 106, 114, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435448_8edb7653b4a4ea560d7c9ef34d033a52_rle_crop_3997860659_0.png', 0, 78, 54, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435448_8edb7653b4a4ea560d7c9ef34d033a52_rle_crop_3997860658_0.png', 0, 72, 153, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435118_f26ac8ffacdcd2ad729565c04e1ed8df_rle_crop_3997860663_0.png', 0, 81, 213, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435116_452f9f38e105388cd170ac11b828a487_rle_crop_3997860664_0.png', 0, 101, 140, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435116_452f9f38e105388cd170ac11b828a487_rle_crop_3997860665_0.png', 0, 103, 131, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435114_5feaa67b597a575ae0609af9ea7802c6_rle_crop_3997860667_0.png', 0, 211, 139, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435112_8bbf7c96d5d53c7b7c796197b5e0974b_rle_crop_3997860670_0.png', 0, 96, 106, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435112_8bbf7c96d5d53c7b7c796197b5e0974b_rle_crop_3997860669_0.png', 0, 148, 122, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435106_40b0f2e2134aae0e9af40fe3b4926623_rle_crop_3997860671_0.png', 0, 137, 88, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435106_40b0f2e2134aae0e9af40fe3b4926623_rle_crop_3997860672_0.png', 0, 148, 117, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435106_40b0f2e2134aae0e9af40fe3b4926623_rle_crop_3997860673_0.png', 0, 62, 78, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389434944_2fa6cbeb15cc9059986c2027d942a8be_rle_crop_3997860674_0.png', 0, 118, 86, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389434943_152d262baca52b6c803328052005587d_rle_crop_3997860676_0.png', 0, 114, 66, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389434935_a1454c9fd8de39193bccdfd584d9f784_rle_crop_3997860679_0.png', 0, 128, 133, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457062), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389434935_a1454c9fd8de39193bccdfd584d9f784_rle_crop_3997860678_0.png', 0, 161, 133, 0, 1760457062,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 32 photos in the portfolio 3736932 time of upload the photos Elapsed time : 10.289542436599731 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3736932 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1760457067_437681 INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457067), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435453_748631fd1d9c7b538d7a8aeadbdeba0b_rle_crop_3997860648_0.png', 0, 81, 106, 0, 1760457067,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457067), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435448_8edb7653b4a4ea560d7c9ef34d033a52_rle_crop_3997860660_0.png', 0, 111, 124, 0, 1760457067,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 2 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.0085277557373047 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! About to upload 6 photos upload in portfolio : 3736932 init cache_photo without model_param we have 6 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1760457071_437681 INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457072), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435118_f26ac8ffacdcd2ad729565c04e1ed8df_rle_crop_3997860662_0.png', 0, 136, 207, 0, 1760457072,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457072), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435118_f26ac8ffacdcd2ad729565c04e1ed8df_rle_crop_3997860661_0.png', 0, 51, 38, 0, 1760457072,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457072), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435114_5feaa67b597a575ae0609af9ea7802c6_rle_crop_3997860668_0.png', 0, 958, 960, 0, 1760457072,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457072), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435114_5feaa67b597a575ae0609af9ea7802c6_rle_crop_3997860666_0.png', 0, 45, 46, 0, 1760457072,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457072), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389434944_2fa6cbeb15cc9059986c2027d942a8be_rle_crop_3997860675_0.png', 0, 146, 154, 0, 1760457072,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457072), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389434943_152d262baca52b6c803328052005587d_rle_crop_3997860677_0.png', 0, 74, 56, 0, 1760457072,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 6 photos in the portfolio 3736932 time of upload the photos Elapsed time : 2.301633834838867 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1760457073_437681 INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760457074), 0.0, 0.0, 14, '', 0, 0, '1760457028_437681_1389435454_ca070e1d964316b528f1bed6e2d2a0ee_rle_crop_3997860645_0.png', 0, 33, 100, 0, 1760457074,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.9390919208526611 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1389435458, 1389435456, 1389435454, 1389435453, 1389435451, 1389435448, 1389435118, 1389435116, 1389435114, 1389435112, 1389435108, 1389435106, 1389434944, 1389434943, 1389434935] Looping around the photos to save general results len do output : 41 /1389448883Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448884Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448885Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448886Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448887Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448888Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448889Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448890Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448891Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448892Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448893Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448894Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448895Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448896Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448897Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448898Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448899Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448900Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448901Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448902Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448903Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448904Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448905Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448906Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448907Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448908Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448909Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448910Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448911Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448912Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448913Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448914Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448915Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448916Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448917Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448918Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448919Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448920Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448921Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448922Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389448923Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435458', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435456', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435454', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435453', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435451', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435448', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435118', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435116', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435114', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435112', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435108', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435106', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389434944', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389434943', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389434935', None, None, None, None, None, '3941696') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 138 time used for this insertion : 0.03838205337524414 save_final save missing photos in datou_result : time spend for datou_step_exec : 20.2980535030365 time spend to save output : 0.0400996208190918 total time spend for step 2 : 20.33815312385559 step3:rle_unique_nms_with_priority Tue Oct 14 17:51:14 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 41 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 2 nb_hashtags : 1 time to prepare the origin masks : 0.049645423889160156 time for calcul the mask position with numpy : 0.09279274940490723 nb_pixel_total : 2050117 time to create 1 rle with new method : 0.07382726669311523 time for calcul the mask position with numpy : 0.006076812744140625 nb_pixel_total : 7105 time to create 1 rle with old method : 0.008175373077392578 time for calcul the mask position with numpy : 0.005726337432861328 nb_pixel_total : 16378 time to create 1 rle with old method : 0.0190126895904541 create new chi : 0.2134721279144287 time to delete rle : 0.10278701782226562 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 1642 TO DO : save crop sub photo not yet done ! save time : 0.3291511535644531 nb_obj : 3 nb_hashtags : 1 time to prepare the origin masks : 0.15068340301513672 time for calcul the mask position with numpy : 0.05237150192260742 nb_pixel_total : 1986836 time to create 1 rle with new method : 0.20250415802001953 time for calcul the mask position with numpy : 0.007300615310668945 nb_pixel_total : 79188 time to create 1 rle with old method : 0.08400750160217285 time for calcul the mask position with numpy : 0.0061016082763671875 nb_pixel_total : 2764 time to create 1 rle with old method : 0.002941608428955078 time for calcul the mask position with numpy : 0.00584101676940918 nb_pixel_total : 4812 time to create 1 rle with old method : 0.005145549774169922 create new chi : 0.37525129318237305 time to delete rle : 0.00029277801513671875 batch 1 Loaded 7 chid ids of type : 3594 ++++Number RLEs to save : 1790 TO DO : save crop sub photo not yet done ! save time : 0.3453240394592285 nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 0.18900775909423828 time for calcul the mask position with numpy : 0.052970170974731445 nb_pixel_total : 2054933 time to create 1 rle with new method : 0.16632342338562012 time for calcul the mask position with numpy : 0.0057048797607421875 nb_pixel_total : 5781 time to create 1 rle with old method : 0.00624847412109375 time for calcul the mask position with numpy : 0.005536794662475586 nb_pixel_total : 3997 time to create 1 rle with old method : 0.0043604373931884766 time for calcul the mask position with numpy : 0.0055310726165771484 nb_pixel_total : 2445 time to create 1 rle with old method : 0.0026383399963378906 time for calcul the mask position with numpy : 0.005876064300537109 nb_pixel_total : 6444 time to create 1 rle with old method : 0.0069086551666259766 create new chi : 0.2710120677947998 time to delete rle : 0.0002567768096923828 batch 1 Loaded 9 chid ids of type : 3594 ++++Number RLEs to save : 1720 TO DO : save crop sub photo not yet done ! save time : 0.33476781845092773 nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 0.06352925300598145 time for calcul the mask position with numpy : 0.052715301513671875 nb_pixel_total : 2047341 time to create 1 rle with new method : 0.15927553176879883 time for calcul the mask position with numpy : 0.0058841705322265625 nb_pixel_total : 6971 time to create 1 rle with old method : 0.007603883743286133 time for calcul the mask position with numpy : 0.0055730342864990234 nb_pixel_total : 10613 time to create 1 rle with old method : 0.011392354965209961 time for calcul the mask position with numpy : 0.005759239196777344 nb_pixel_total : 2239 time to create 1 rle with old method : 0.002351999282836914 time for calcul the mask position with numpy : 0.005671024322509766 nb_pixel_total : 6436 time to create 1 rle with old method : 0.0067615509033203125 create new chi : 0.27077531814575195 time to delete rle : 0.00029277801513671875 batch 1 Loaded 9 chid ids of type : 3594 ++++Number RLEs to save : 1864 TO DO : save crop sub photo not yet done ! save time : 0.33954286575317383 nb_obj : 5 nb_hashtags : 1 time to prepare the origin masks : 0.07599353790283203 time for calcul the mask position with numpy : 0.1101081371307373 nb_pixel_total : 2011536 time to create 1 rle with new method : 0.07019901275634766 time for calcul the mask position with numpy : 0.005927324295043945 nb_pixel_total : 3439 time to create 1 rle with old method : 0.0036897659301757812 time for calcul the mask position with numpy : 0.005651712417602539 nb_pixel_total : 6723 time to create 1 rle with old method : 0.007077217102050781 time for calcul the mask position with numpy : 0.005542755126953125 nb_pixel_total : 8686 time to create 1 rle with old method : 0.009269475936889648 time for calcul the mask position with numpy : 0.005795001983642578 nb_pixel_total : 38949 time to create 1 rle with old method : 0.04200267791748047 time for calcul the mask position with numpy : 0.00608372688293457 nb_pixel_total : 4267 time to create 1 rle with old method : 0.0047414302825927734 create new chi : 0.28526878356933594 time to delete rle : 0.0003459453582763672 batch 1 Loaded 12 chid ids of type : 3594 +++++++Number RLEs to save : 2188 TO DO : save crop sub photo not yet done ! save time : 0.37937140464782715 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.04795718193054199 time for calcul the mask position with numpy : 0.07743549346923828 nb_pixel_total : 2054882 time to create 1 rle with new method : 0.07188177108764648 time for calcul the mask position with numpy : 0.005978822708129883 nb_pixel_total : 10262 time to create 1 rle with old method : 0.011283397674560547 time for calcul the mask position with numpy : 0.005999326705932617 nb_pixel_total : 2513 time to create 1 rle with old method : 0.0029599666595458984 time for calcul the mask position with numpy : 0.005705356597900391 nb_pixel_total : 5943 time to create 1 rle with old method : 0.006650447845458984 create new chi : 0.19795489311218262 time to delete rle : 0.00025773048400878906 batch 1 Loaded 7 chid ids of type : 3594 +++Number RLEs to save : 1744 TO DO : save crop sub photo not yet done ! save time : 0.31812572479248047 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.04842257499694824 time for calcul the mask position with numpy : 0.10145378112792969 nb_pixel_total : 2038290 time to create 1 rle with new method : 0.21556329727172852 time for calcul the mask position with numpy : 0.005930900573730469 nb_pixel_total : 10895 time to create 1 rle with old method : 0.012000322341918945 time for calcul the mask position with numpy : 0.005984306335449219 nb_pixel_total : 22839 time to create 1 rle with old method : 0.02442002296447754 time for calcul the mask position with numpy : 0.005750179290771484 nb_pixel_total : 1576 time to create 1 rle with old method : 0.0017867088317871094 create new chi : 0.38164758682250977 time to delete rle : 0.00032806396484375 batch 1 Loaded 7 chid ids of type : 3594 ++++Number RLEs to save : 2008 TO DO : save crop sub photo not yet done ! save time : 0.3912034034729004 nb_obj : 2 nb_hashtags : 1 time to prepare the origin masks : 0.04770493507385254 time for calcul the mask position with numpy : 0.019005537033081055 nb_pixel_total : 2052376 time to create 1 rle with new method : 0.07185912132263184 time for calcul the mask position with numpy : 0.005793571472167969 nb_pixel_total : 10811 time to create 1 rle with old method : 0.011810064315795898 time for calcul the mask position with numpy : 0.00556182861328125 nb_pixel_total : 10413 time to create 1 rle with old method : 0.011347293853759766 create new chi : 0.13201403617858887 time to delete rle : 0.0002658367156982422 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 1626 TO DO : save crop sub photo not yet done ! save time : 0.3157963752746582 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.06787610054016113 time for calcul the mask position with numpy : 0.03303813934326172 nb_pixel_total : 1410831 time to create 1 rle with new method : 0.08939886093139648 time for calcul the mask position with numpy : 0.011056900024414062 nb_pixel_total : 646994 time to create 1 rle with new method : 0.17541861534118652 time for calcul the mask position with numpy : 0.005974531173706055 nb_pixel_total : 13996 time to create 1 rle with old method : 0.015799283981323242 time for calcul the mask position with numpy : 0.006073951721191406 nb_pixel_total : 1779 time to create 1 rle with old method : 0.001968860626220703 create new chi : 0.34728407859802246 time to delete rle : 0.0005106925964355469 batch 1 Loaded 7 chid ids of type : 3594 ++++Number RLEs to save : 4102 TO DO : save crop sub photo not yet done ! save time : 0.5899198055267334 nb_obj : 2 nb_hashtags : 1 time to prepare the origin masks : 0.04095268249511719 time for calcul the mask position with numpy : 0.018112897872924805 nb_pixel_total : 2055654 time to create 1 rle with new method : 0.04445242881774902 time for calcul the mask position with numpy : 0.005846261978149414 nb_pixel_total : 6073 time to create 1 rle with old method : 0.006490230560302734 time for calcul the mask position with numpy : 0.005556583404541016 nb_pixel_total : 11873 time to create 1 rle with old method : 0.012590646743774414 create new chi : 0.09971928596496582 time to delete rle : 0.0002276897430419922 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 1538 TO DO : save crop sub photo not yet done ! save time : 0.31088733673095703 No data in photo_id : 1389435108 nb_obj : 3 nb_hashtags : 1 time to prepare the origin masks : 0.05283832550048828 time for calcul the mask position with numpy : 0.042769432067871094 nb_pixel_total : 2053058 time to create 1 rle with new method : 0.17042875289916992 time for calcul the mask position with numpy : 0.005927085876464844 nb_pixel_total : 2324 time to create 1 rle with old method : 0.0025725364685058594 time for calcul the mask position with numpy : 0.0056874752044677734 nb_pixel_total : 11082 time to create 1 rle with old method : 0.01256108283996582 time for calcul the mask position with numpy : 0.00590062141418457 nb_pixel_total : 7136 time to create 1 rle with old method : 0.00779414176940918 create new chi : 0.2635173797607422 time to delete rle : 0.0002779960632324219 batch 1 Loaded 7 chid ids of type : 3594 +++++++Number RLEs to save : 1812 TO DO : save crop sub photo not yet done ! save time : 0.319866418838501 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.03887820243835449 time for calcul the mask position with numpy : 0.10939550399780273 nb_pixel_total : 2051070 time to create 1 rle with new method : 0.08678889274597168 time for calcul the mask position with numpy : 0.0058896541595458984 nb_pixel_total : 16790 time to create 1 rle with old method : 0.018894195556640625 time for calcul the mask position with numpy : 0.00613856315612793 nb_pixel_total : 5740 time to create 1 rle with old method : 0.0065996646881103516 create new chi : 0.24418306350708008 time to delete rle : 0.00025272369384765625 batch 1 Loaded 5 chid ids of type : 3594 +++Number RLEs to save : 1586 TO DO : save crop sub photo not yet done ! save time : 0.3146328926086426 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.0379335880279541 time for calcul the mask position with numpy : 0.11157560348510742 nb_pixel_total : 2064222 time to create 1 rle with new method : 0.14390873908996582 time for calcul the mask position with numpy : 0.005964040756225586 nb_pixel_total : 3068 time to create 1 rle with old method : 0.003421783447265625 time for calcul the mask position with numpy : 0.005863666534423828 nb_pixel_total : 6310 time to create 1 rle with old method : 0.007164955139160156 create new chi : 0.28727197647094727 time to delete rle : 0.0002262592315673828 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 1324 TO DO : save crop sub photo not yet done ! save time : 0.2730269432067871 nb_obj : 2 nb_hashtags : 1 time to prepare the origin masks : 0.04583430290222168 time for calcul the mask position with numpy : 0.041126251220703125 nb_pixel_total : 2054873 time to create 1 rle with new method : 0.07989335060119629 time for calcul the mask position with numpy : 0.006180286407470703 nb_pixel_total : 6165 time to create 1 rle with old method : 0.0068552494049072266 time for calcul the mask position with numpy : 0.005921363830566406 nb_pixel_total : 12562 time to create 1 rle with old method : 0.014320611953735352 create new chi : 0.16454410552978516 time to delete rle : 0.0002548694610595703 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 1616 TO DO : save crop sub photo not yet done ! save time : 0.3130619525909424 map_output_result : {1389435458: (0.0, 'Should be the crop_list due to order', 0), 1389435456: (0.0, 'Should be the crop_list due to order', 0), 1389435454: (0.0, 'Should be the crop_list due to order', 0), 1389435453: (0.0, 'Should be the crop_list due to order', 0), 1389435451: (0.0, 'Should be the crop_list due to order', 0), 1389435448: (0.0, 'Should be the crop_list due to order', 0), 1389435118: (0.0, 'Should be the crop_list due to order', 0), 1389435116: (0.0, 'Should be the crop_list due to order', 0), 1389435114: (0.0, 'Should be the crop_list due to order', 0), 1389435112: (0.0, 'Should be the crop_list due to order', 0), 1389435108: (0.0, 'Should be the crop_list due to order', 0.0), 1389435106: (0.0, 'Should be the crop_list due to order', 0), 1389434944: (0.0, 'Should be the crop_list due to order', 0), 1389434943: (0.0, 'Should be the crop_list due to order', 0), 1389434935: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1389435458, 1389435456, 1389435454, 1389435453, 1389435451, 1389435448, 1389435118, 1389435116, 1389435114, 1389435112, 1389435108, 1389435106, 1389434944, 1389434943, 1389434935] Looping around the photos to save general results len do output : 15 /1389435458.Didn't retrieve data . /1389435456.Didn't retrieve data . /1389435454.Didn't retrieve data . /1389435453.Didn't retrieve data . /1389435451.Didn't retrieve data . /1389435448.Didn't retrieve data . /1389435118.Didn't retrieve data . /1389435116.Didn't retrieve data . /1389435114.Didn't retrieve data . /1389435112.Didn't retrieve data . /1389435108.Didn't retrieve data . /1389435106.Didn't retrieve data . /1389434944.Didn't retrieve data . /1389434943.Didn't retrieve data . /1389434935.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435458', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435456', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435454', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435453', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435451', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435448', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435118', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435116', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435114', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435112', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435108', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435106', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389434944', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389434943', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389434935', None, None, None, None, None, '3941696') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 45 time used for this insertion : 0.03660321235656738 save_final save missing photos in datou_result : time spend for datou_step_exec : 10.02089262008667 time spend to save output : 0.0372469425201416 total time spend for step 3 : 10.058139562606812 step4:ventilate_hashtags_in_portfolio Tue Oct 14 17:51:24 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 27805408 get user id for portfolio 27805408 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27805408 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('environnement','pet_clair','pehd','carton','background','autre','metal','pet_fonce','mal_croppe','flou','papier')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27805408 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('environnement','pet_clair','pehd','carton','background','autre','metal','pet_fonce','mal_croppe','flou','papier')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27805408 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('environnement','pet_clair','pehd','carton','background','autre','metal','pet_fonce','mal_croppe','flou','papier')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://marlene.fotonower.com/velours/27805797,27805798,27805799,27805800,27805801,27805802,27805803,27805804,27805805,27805806,27805807?tags=environnement,pet_clair,pehd,carton,background,autre,metal,pet_fonce,mal_croppe,flou,papier Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1389435458, 1389435456, 1389435454, 1389435453, 1389435451, 1389435448, 1389435118, 1389435116, 1389435114, 1389435112, 1389435108, 1389435106, 1389434944, 1389434943, 1389434935] Looping around the photos to save general results len do output : 1 /27805408. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435458', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435456', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435454', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435453', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435451', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435448', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435118', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435116', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435114', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435112', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435108', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435106', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389434944', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389434943', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389434935', None, None, None, None, None, '3941696') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 16 time used for this insertion : 0.037899017333984375 save_final save missing photos in datou_result : time spend for datou_step_exec : 5.485337018966675 time spend to save output : 0.03825116157531738 total time spend for step 4 : 5.523588180541992 step5:final Tue Oct 14 17:51:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1389435458: ('0.03357706404320987',), 1389435456: ('0.03357706404320987',), 1389435454: ('0.03357706404320987',), 1389435453: ('0.03357706404320987',), 1389435451: ('0.03357706404320987',), 1389435448: ('0.03357706404320987',), 1389435118: ('0.03357706404320987',), 1389435116: ('0.03357706404320987',), 1389435114: ('0.03357706404320987',), 1389435112: ('0.03357706404320987',), 1389435108: ('0.03357706404320987',), 1389435106: ('0.03357706404320987',), 1389434944: ('0.03357706404320987',), 1389434943: ('0.03357706404320987',), 1389434935: ('0.03357706404320987',)} new output for save of step final : {1389435458: ('0.03357706404320987',), 1389435456: ('0.03357706404320987',), 1389435454: ('0.03357706404320987',), 1389435453: ('0.03357706404320987',), 1389435451: ('0.03357706404320987',), 1389435448: ('0.03357706404320987',), 1389435118: ('0.03357706404320987',), 1389435116: ('0.03357706404320987',), 1389435114: ('0.03357706404320987',), 1389435112: ('0.03357706404320987',), 1389435108: ('0.03357706404320987',), 1389435106: ('0.03357706404320987',), 1389434944: ('0.03357706404320987',), 1389434943: ('0.03357706404320987',), 1389434935: ('0.03357706404320987',)} [1389435458, 1389435456, 1389435454, 1389435453, 1389435451, 1389435448, 1389435118, 1389435116, 1389435114, 1389435112, 1389435108, 1389435106, 1389434944, 1389434943, 1389434935] Looping around the photos to save general results len do output : 15 /1389435458.Didn't retrieve data . /1389435456.Didn't retrieve data . /1389435454.Didn't retrieve data . /1389435453.Didn't retrieve data . /1389435451.Didn't retrieve data . /1389435448.Didn't retrieve data . /1389435118.Didn't retrieve data . /1389435116.Didn't retrieve data . /1389435114.Didn't retrieve data . /1389435112.Didn't retrieve data . /1389435108.Didn't retrieve data . /1389435106.Didn't retrieve data . /1389434944.Didn't retrieve data . /1389434943.Didn't retrieve data . /1389434935.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435458', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435456', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435454', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435453', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435451', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435448', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435118', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435116', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435114', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435112', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435108', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435106', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389434944', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389434943', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389434935', None, None, None, None, None, '3941696') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 45 time used for this insertion : 0.03661990165710449 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.375072717666626 time spend to save output : 0.03729867935180664 total time spend for step 5 : 0.4123713970184326 step6:blur_detection Tue Oct 14 17:51:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1760457028_437681_1389435458_1711d6fb738f9979c690a0002136a339.jpg resize: (1080, 1920) 1389435458 -0.8922751948360783 treat image : temp/1760457028_437681_1389435456_d5694014c6fe3c5607f85e1155e2ff51.jpg resize: (1080, 1920) 1389435456 -1.8228711557047508 treat image : temp/1760457028_437681_1389435454_ca070e1d964316b528f1bed6e2d2a0ee.jpg resize: (1080, 1920) 1389435454 -2.3087344371564815 treat image : temp/1760457028_437681_1389435453_748631fd1d9c7b538d7a8aeadbdeba0b.jpg resize: (1080, 1920) 1389435453 -1.5719783042574884 treat image : temp/1760457028_437681_1389435451_73d2f19f8212a6892ca6e54b07610b48.jpg resize: (1080, 1920) 1389435451 -0.5440758161456372 treat image : temp/1760457028_437681_1389435448_8edb7653b4a4ea560d7c9ef34d033a52.jpg resize: (1080, 1920) 1389435448 1.1968598165893738 treat image : temp/1760457028_437681_1389435118_f26ac8ffacdcd2ad729565c04e1ed8df.jpg resize: (1080, 1920) 1389435118 0.20575250708397624 treat image : temp/1760457028_437681_1389435116_452f9f38e105388cd170ac11b828a487.jpg resize: (1080, 1920) 1389435116 0.011774985674546481 treat image : temp/1760457028_437681_1389435114_5feaa67b597a575ae0609af9ea7802c6.jpg resize: (1080, 1920) 1389435114 0.04932538512325038 treat image : temp/1760457028_437681_1389435112_8bbf7c96d5d53c7b7c796197b5e0974b.jpg resize: (1080, 1920) 1389435112 0.45535248952233887 treat image : temp/1760457028_437681_1389435108_66a49dfa38ab41d9a9f6ad92ce36093c.jpg resize: (1080, 1920) 1389435108 -0.2329363159933889 treat image : temp/1760457028_437681_1389435106_40b0f2e2134aae0e9af40fe3b4926623.jpg resize: (1080, 1920) 1389435106 -1.2813820895473054 treat image : temp/1760457028_437681_1389434944_2fa6cbeb15cc9059986c2027d942a8be.jpg resize: (1080, 1920) 1389434944 2.9392726667060702 treat image : temp/1760457028_437681_1389434943_152d262baca52b6c803328052005587d.jpg resize: (1080, 1920) 1389434943 6.511923503051604 treat image : temp/1760457028_437681_1389434935_a1454c9fd8de39193bccdfd584d9f784.jpg resize: (1080, 1920) 1389434935 -0.4269925565150518 treat image : temp/1760457028_437681_1389435458_1711d6fb738f9979c690a0002136a339_rle_crop_3997860639_0.png resize: (130, 176) 1389448883 0.9031232186151046 treat image : temp/1760457028_437681_1389435458_1711d6fb738f9979c690a0002136a339_rle_crop_3997860640_0.png resize: (151, 85) 1389448884 -1.92024888182492 treat image : temp/1760457028_437681_1389435456_d5694014c6fe3c5607f85e1155e2ff51_rle_crop_3997860643_0.png resize: (212, 552) 1389448885 -2.1305632608921266 treat image : temp/1760457028_437681_1389435456_d5694014c6fe3c5607f85e1155e2ff51_rle_crop_3997860642_0.png resize: (46, 70) 1389448886 2.3380484568208315 treat image : temp/1760457028_437681_1389435456_d5694014c6fe3c5607f85e1155e2ff51_rle_crop_3997860641_0.png resize: (69, 99) 1389448887 -3.0931516710061757 treat image : temp/1760457028_437681_1389435454_ca070e1d964316b528f1bed6e2d2a0ee_rle_crop_3997860646_0.png resize: (47, 116) 1389448888 -2.1319948063836964 treat image : temp/1760457028_437681_1389435454_ca070e1d964316b528f1bed6e2d2a0ee_rle_crop_3997860647_0.png resize: (78, 104) 1389448889 -0.7041051079467904 treat image : temp/1760457028_437681_1389435454_ca070e1d964316b528f1bed6e2d2a0ee_rle_crop_3997860644_0.png resize: (83, 114) 1389448890 -2.468201242930411 treat image : temp/1760457028_437681_1389435453_748631fd1d9c7b538d7a8aeadbdeba0b_rle_crop_3997860651_0.png resize: (103, 123) 1389448891 -1.8951673521888344 treat image : temp/1760457028_437681_1389435453_748631fd1d9c7b538d7a8aeadbdeba0b_rle_crop_3997860649_0.png resize: (51, 56) 1389448892 -0.20561276497242192 treat image : temp/1760457028_437681_1389435453_748631fd1d9c7b538d7a8aeadbdeba0b_rle_crop_3997860650_0.png resize: (130, 116) 1389448893 -1.4194029723302493 treat image : temp/1760457028_437681_1389435451_73d2f19f8212a6892ca6e54b07610b48_rle_crop_3997860653_0.png resize: (155, 340) 1389448894 -2.304020211738404 treat image : temp/1760457028_437681_1389435451_73d2f19f8212a6892ca6e54b07610b48_rle_crop_3997860654_0.png resize: (68, 94) 1389448895 -1.4398780788583587 treat image : temp/1760457028_437681_1389435451_73d2f19f8212a6892ca6e54b07610b48_rle_crop_3997860656_0.png resize: (122, 74) 1389448896 1.159920443200292 treat image : temp/1760457028_437681_1389435451_73d2f19f8212a6892ca6e54b07610b48_rle_crop_3997860657_0.png resize: (61, 72) 1389448897 -2.3480910060915927 treat image : temp/1760457028_437681_1389435451_73d2f19f8212a6892ca6e54b07610b48_rle_crop_3997860652_0.png resize: (70, 78) 1389448898 0.10034518474569847 treat image : temp/1760457028_437681_1389435451_73d2f19f8212a6892ca6e54b07610b48_rle_crop_3997860655_0.png resize: (114, 106) 1389448899 -2.6899760554304133 treat image : temp/1760457028_437681_1389435448_8edb7653b4a4ea560d7c9ef34d033a52_rle_crop_3997860659_0.png resize: (54, 78) 1389448900 -3.5454424084138316 treat image : temp/1760457028_437681_1389435448_8edb7653b4a4ea560d7c9ef34d033a52_rle_crop_3997860658_0.png resize: (153, 72) 1389448901 -1.0547126731493381 treat image : temp/1760457028_437681_1389435118_f26ac8ffacdcd2ad729565c04e1ed8df_rle_crop_3997860663_0.png resize: (213, 81) 1389448902 -1.547017067666683 treat image : temp/1760457028_437681_1389435116_452f9f38e105388cd170ac11b828a487_rle_crop_3997860664_0.png resize: (140, 101) 1389448903 -3.565445411486254 treat image : temp/1760457028_437681_1389435116_452f9f38e105388cd170ac11b828a487_rle_crop_3997860665_0.png resize: (131, 103) 1389448904 0.3857003605548037 treat image : temp/1760457028_437681_1389435114_5feaa67b597a575ae0609af9ea7802c6_rle_crop_3997860667_0.png resize: (139, 211) 1389448905 -0.43581828947815154 treat image : temp/1760457028_437681_1389435112_8bbf7c96d5d53c7b7c796197b5e0974b_rle_crop_3997860670_0.png resize: (106, 96) 1389448906 0.046570590474113584 treat image : temp/1760457028_437681_1389435112_8bbf7c96d5d53c7b7c796197b5e0974b_rle_crop_3997860669_0.png resize: (122, 148) 1389448907 -0.68039768199375 treat image : temp/1760457028_437681_1389435106_40b0f2e2134aae0e9af40fe3b4926623_rle_crop_3997860671_0.png resize: (88, 137) 1389448908 -3.278456893271659 treat image : temp/1760457028_437681_1389435106_40b0f2e2134aae0e9af40fe3b4926623_rle_crop_3997860672_0.png resize: (117, 148) 1389448909 -1.711440303648098 treat image : temp/1760457028_437681_1389435106_40b0f2e2134aae0e9af40fe3b4926623_rle_crop_3997860673_0.png resize: (78, 62) 1389448910 -3.098822435816302 treat image : temp/1760457028_437681_1389434944_2fa6cbeb15cc9059986c2027d942a8be_rle_crop_3997860674_0.png resize: (86, 118) 1389448911 3.074778909256397 treat image : temp/1760457028_437681_1389434943_152d262baca52b6c803328052005587d_rle_crop_3997860676_0.png resize: (66, 114) 1389448912 6.2911317169068 treat image : temp/1760457028_437681_1389434935_a1454c9fd8de39193bccdfd584d9f784_rle_crop_3997860679_0.png resize: (133, 128) 1389448913 0.17958421549390774 treat image : temp/1760457028_437681_1389434935_a1454c9fd8de39193bccdfd584d9f784_rle_crop_3997860678_0.png resize: (133, 161) 1389448914 -2.011094826577025 treat image : temp/1760457028_437681_1389435453_748631fd1d9c7b538d7a8aeadbdeba0b_rle_crop_3997860648_0.png resize: (106, 81) 1389448915 0.6973469585208779 treat image : temp/1760457028_437681_1389435448_8edb7653b4a4ea560d7c9ef34d033a52_rle_crop_3997860660_0.png resize: (124, 111) 1389448916 -0.9227430862599696 treat image : temp/1760457028_437681_1389435118_f26ac8ffacdcd2ad729565c04e1ed8df_rle_crop_3997860662_0.png resize: (207, 136) 1389448917 0.544302862107456 treat image : temp/1760457028_437681_1389435118_f26ac8ffacdcd2ad729565c04e1ed8df_rle_crop_3997860661_0.png resize: (38, 51) 1389448918 20.0 treat image : temp/1760457028_437681_1389435114_5feaa67b597a575ae0609af9ea7802c6_rle_crop_3997860668_0.png resize: (960, 958) 1389448919 0.33696471227995795 treat image : temp/1760457028_437681_1389435114_5feaa67b597a575ae0609af9ea7802c6_rle_crop_3997860666_0.png resize: (46, 45) 1389448920 -3.3369042031675122 treat image : temp/1760457028_437681_1389434944_2fa6cbeb15cc9059986c2027d942a8be_rle_crop_3997860675_0.png resize: (154, 146) 1389448921 0.3097421541225001 treat image : temp/1760457028_437681_1389434943_152d262baca52b6c803328052005587d_rle_crop_3997860677_0.png resize: (56, 74) 1389448922 0.6072263632376738 treat image : temp/1760457028_437681_1389435454_ca070e1d964316b528f1bed6e2d2a0ee_rle_crop_3997860645_0.png resize: (100, 33) 1389448923 2.2312932177560163 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 56 time used for this insertion : 0.03705620765686035 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 56 time used for this insertion : 0.03716611862182617 save missing photos in datou_result : time spend for datou_step_exec : 12.125829458236694 time spend to save output : 0.09193563461303711 total time spend for step 6 : 12.217765092849731 step7:brightness Tue Oct 14 17:51:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1760457028_437681_1389435458_1711d6fb738f9979c690a0002136a339.jpg treat image : temp/1760457028_437681_1389435456_d5694014c6fe3c5607f85e1155e2ff51.jpg treat image : temp/1760457028_437681_1389435454_ca070e1d964316b528f1bed6e2d2a0ee.jpg treat image : temp/1760457028_437681_1389435453_748631fd1d9c7b538d7a8aeadbdeba0b.jpg treat image : temp/1760457028_437681_1389435451_73d2f19f8212a6892ca6e54b07610b48.jpg treat image : temp/1760457028_437681_1389435448_8edb7653b4a4ea560d7c9ef34d033a52.jpg treat image : temp/1760457028_437681_1389435118_f26ac8ffacdcd2ad729565c04e1ed8df.jpg treat image : temp/1760457028_437681_1389435116_452f9f38e105388cd170ac11b828a487.jpg treat image : temp/1760457028_437681_1389435114_5feaa67b597a575ae0609af9ea7802c6.jpg treat image : temp/1760457028_437681_1389435112_8bbf7c96d5d53c7b7c796197b5e0974b.jpg treat image : temp/1760457028_437681_1389435108_66a49dfa38ab41d9a9f6ad92ce36093c.jpg treat image : temp/1760457028_437681_1389435106_40b0f2e2134aae0e9af40fe3b4926623.jpg treat image : temp/1760457028_437681_1389434944_2fa6cbeb15cc9059986c2027d942a8be.jpg treat image : temp/1760457028_437681_1389434943_152d262baca52b6c803328052005587d.jpg treat image : temp/1760457028_437681_1389434935_a1454c9fd8de39193bccdfd584d9f784.jpg treat image : temp/1760457028_437681_1389435458_1711d6fb738f9979c690a0002136a339_rle_crop_3997860639_0.png treat image : temp/1760457028_437681_1389435458_1711d6fb738f9979c690a0002136a339_rle_crop_3997860640_0.png treat image : temp/1760457028_437681_1389435456_d5694014c6fe3c5607f85e1155e2ff51_rle_crop_3997860643_0.png treat image : temp/1760457028_437681_1389435456_d5694014c6fe3c5607f85e1155e2ff51_rle_crop_3997860642_0.png treat image : temp/1760457028_437681_1389435456_d5694014c6fe3c5607f85e1155e2ff51_rle_crop_3997860641_0.png treat image : temp/1760457028_437681_1389435454_ca070e1d964316b528f1bed6e2d2a0ee_rle_crop_3997860646_0.png treat image : temp/1760457028_437681_1389435454_ca070e1d964316b528f1bed6e2d2a0ee_rle_crop_3997860647_0.png treat image : temp/1760457028_437681_1389435454_ca070e1d964316b528f1bed6e2d2a0ee_rle_crop_3997860644_0.png treat image : temp/1760457028_437681_1389435453_748631fd1d9c7b538d7a8aeadbdeba0b_rle_crop_3997860651_0.png treat image : temp/1760457028_437681_1389435453_748631fd1d9c7b538d7a8aeadbdeba0b_rle_crop_3997860649_0.png treat image : temp/1760457028_437681_1389435453_748631fd1d9c7b538d7a8aeadbdeba0b_rle_crop_3997860650_0.png treat image : temp/1760457028_437681_1389435451_73d2f19f8212a6892ca6e54b07610b48_rle_crop_3997860653_0.png treat image : temp/1760457028_437681_1389435451_73d2f19f8212a6892ca6e54b07610b48_rle_crop_3997860654_0.png treat image : temp/1760457028_437681_1389435451_73d2f19f8212a6892ca6e54b07610b48_rle_crop_3997860656_0.png treat image : temp/1760457028_437681_1389435451_73d2f19f8212a6892ca6e54b07610b48_rle_crop_3997860657_0.png treat image : temp/1760457028_437681_1389435451_73d2f19f8212a6892ca6e54b07610b48_rle_crop_3997860652_0.png treat image : temp/1760457028_437681_1389435451_73d2f19f8212a6892ca6e54b07610b48_rle_crop_3997860655_0.png treat image : temp/1760457028_437681_1389435448_8edb7653b4a4ea560d7c9ef34d033a52_rle_crop_3997860659_0.png treat image : temp/1760457028_437681_1389435448_8edb7653b4a4ea560d7c9ef34d033a52_rle_crop_3997860658_0.png treat image : temp/1760457028_437681_1389435118_f26ac8ffacdcd2ad729565c04e1ed8df_rle_crop_3997860663_0.png treat image : temp/1760457028_437681_1389435116_452f9f38e105388cd170ac11b828a487_rle_crop_3997860664_0.png treat image : temp/1760457028_437681_1389435116_452f9f38e105388cd170ac11b828a487_rle_crop_3997860665_0.png treat image : temp/1760457028_437681_1389435114_5feaa67b597a575ae0609af9ea7802c6_rle_crop_3997860667_0.png treat image : temp/1760457028_437681_1389435112_8bbf7c96d5d53c7b7c796197b5e0974b_rle_crop_3997860670_0.png treat image : temp/1760457028_437681_1389435112_8bbf7c96d5d53c7b7c796197b5e0974b_rle_crop_3997860669_0.png treat image : temp/1760457028_437681_1389435106_40b0f2e2134aae0e9af40fe3b4926623_rle_crop_3997860671_0.png treat image : temp/1760457028_437681_1389435106_40b0f2e2134aae0e9af40fe3b4926623_rle_crop_3997860672_0.png treat image : temp/1760457028_437681_1389435106_40b0f2e2134aae0e9af40fe3b4926623_rle_crop_3997860673_0.png treat image : temp/1760457028_437681_1389434944_2fa6cbeb15cc9059986c2027d942a8be_rle_crop_3997860674_0.png treat image : temp/1760457028_437681_1389434943_152d262baca52b6c803328052005587d_rle_crop_3997860676_0.png treat image : temp/1760457028_437681_1389434935_a1454c9fd8de39193bccdfd584d9f784_rle_crop_3997860679_0.png treat image : temp/1760457028_437681_1389434935_a1454c9fd8de39193bccdfd584d9f784_rle_crop_3997860678_0.png treat image : temp/1760457028_437681_1389435453_748631fd1d9c7b538d7a8aeadbdeba0b_rle_crop_3997860648_0.png treat image : temp/1760457028_437681_1389435448_8edb7653b4a4ea560d7c9ef34d033a52_rle_crop_3997860660_0.png treat image : temp/1760457028_437681_1389435118_f26ac8ffacdcd2ad729565c04e1ed8df_rle_crop_3997860662_0.png treat image : temp/1760457028_437681_1389435118_f26ac8ffacdcd2ad729565c04e1ed8df_rle_crop_3997860661_0.png treat image : temp/1760457028_437681_1389435114_5feaa67b597a575ae0609af9ea7802c6_rle_crop_3997860668_0.png treat image : temp/1760457028_437681_1389435114_5feaa67b597a575ae0609af9ea7802c6_rle_crop_3997860666_0.png treat image : temp/1760457028_437681_1389434944_2fa6cbeb15cc9059986c2027d942a8be_rle_crop_3997860675_0.png treat image : temp/1760457028_437681_1389434943_152d262baca52b6c803328052005587d_rle_crop_3997860677_0.png treat image : temp/1760457028_437681_1389435454_ca070e1d964316b528f1bed6e2d2a0ee_rle_crop_3997860645_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 56 time used for this insertion : 0.0387873649597168 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 56 time used for this insertion : 0.0368192195892334 save missing photos in datou_result : time spend for datou_step_exec : 3.5065364837646484 time spend to save output : 0.09310126304626465 total time spend for step 7 : 3.599637746810913 step8:velours_tree Tue Oct 14 17:51:46 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.1299276351928711 time spend to save output : 3.170967102050781e-05 total time spend for step 8 : 0.1299593448638916 step9:send_mail_cod Tue Oct 14 17:51:46 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P27805408_14-10-2025_17_51_46.pdf 27805798 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette278057981760457106 27805799 imagette278057991760457106 27805800 change filename to text .change filename to text .imagette278058001760457106 27805801 imagette278058011760457106 27805802 change filename to text .imagette278058021760457107 27805803 imagette278058031760457107 27805804 imagette278058041760457107 27805805 imagette278058051760457107 27805806 imagette278058061760457107 27805807 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette278058071760457107 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=27805408 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://marlene.fotonower.com/velours/27805797,27805798,27805799,27805800,27805801,27805802,27805803,27805804,27805805,27805806,27805807?tags=environnement,pet_clair,pehd,carton,background,autre,metal,pet_fonce,mal_croppe,flou,papier args[1389435458] : ((1389435458, -0.8922751948360783, 492688767), (1389435458, 0.5238517813956608, 2107752395), '0.03357706404320987') We are sending mail with results at report@fotonower.com args[1389435456] : ((1389435456, -1.8228711557047508, 492688767), (1389435456, 0.32542364068754037, 2107752395), '0.03357706404320987') We are sending mail with results at report@fotonower.com args[1389435454] : ((1389435454, -2.3087344371564815, 492609224), (1389435454, 0.31862828682194605, 2107752395), '0.03357706404320987') We are sending mail with results at report@fotonower.com args[1389435453] : ((1389435453, -1.5719783042574884, 492688767), (1389435453, 0.23749628030920703, 2107752395), '0.03357706404320987') We are sending mail with results at report@fotonower.com args[1389435451] : ((1389435451, -0.5440758161456372, 492688767), (1389435451, 0.1701970779391327, 2107752395), '0.03357706404320987') We are sending mail with results at report@fotonower.com args[1389435448] : ((1389435448, 1.1968598165893738, 492688767), (1389435448, 0.3067645187193945, 2107752395), '0.03357706404320987') We are sending mail with results at report@fotonower.com args[1389435118] : ((1389435118, 0.20575250708397624, 492688767), (1389435118, 0.6465292395659017, 2107752395), '0.03357706404320987') We are sending mail with results at report@fotonower.com args[1389435116] : ((1389435116, 0.011774985674546481, 492688767), (1389435116, 0.7231785460066442, 2107752395), '0.03357706404320987') We are sending mail with results at report@fotonower.com args[1389435114] : ((1389435114, 0.04932538512325038, 492688767), (1389435114, 0.8100986933615673, 2107752395), '0.03357706404320987') We are sending mail with results at report@fotonower.com args[1389435112] : ((1389435112, 0.45535248952233887, 492688767), (1389435112, 0.553847277883607, 2107752395), '0.03357706404320987') We are sending mail with results at report@fotonower.com args[1389435108] : ((1389435108, -0.2329363159933889, 492688767), (1389435108, 1.1576959372907263, 2107752395), '0.03357706404320987') We are sending mail with results at report@fotonower.com args[1389435106] : ((1389435106, -1.2813820895473054, 492688767), (1389435106, 0.5698314063060655, 2107752395), '0.03357706404320987') We are sending mail with results at report@fotonower.com args[1389434944] : ((1389434944, 2.9392726667060702, 492688767), (1389434944, 0.7268580221650961, 2107752395), '0.03357706404320987') We are sending mail with results at report@fotonower.com args[1389434943] : ((1389434943, 6.511923503051604, 492688767), (1389434943, 0.755317683539657, 2107752395), '0.03357706404320987') We are sending mail with results at report@fotonower.com args[1389434935] : ((1389434935, -0.4269925565150518, 492688767), (1389434935, 1.14644769148913, 2107752395), '0.03357706404320987') We are sending mail with results at report@fotonower.com refus_total : 0.03357706404320987 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos_view ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=27805408 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27805408_14-10-2025_17_51_46.pdf results_Auto_P27805408_14-10-2025_17_51_46.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27805408_14-10-2025_17_51_46.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','27805408','results_Auto_P27805408_14-10-2025_17_51_46.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27805408_14-10-2025_17_51_46.pdf','pdf','','0.21','0.03357706404320987') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/27805408

https://www.fotonower.com/image?json=false&list_photos_id=1389435458
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1389435456
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1389435454
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1389435453
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1389435451
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1389435448
La photo est trop floue, merci de reprendre une photo.(avec le score = 1.1968598165893738)
https://www.fotonower.com/image?json=false&list_photos_id=1389435118
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1389435116
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1389435114
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1389435112
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1389435108
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1389435106
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1389434944
La photo est trop floue, merci de reprendre une photo.(avec le score = 2.9392726667060702)
https://www.fotonower.com/image?json=false&list_photos_id=1389434943
La photo est trop floue, merci de reprendre une photo.(avec le score = 6.511923503051604)
https://www.fotonower.com/image?json=false&list_photos_id=1389434935
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 3.36%
Veuillez trouver les photos des contaminants.

exemples de contaminants: pet_clair: https://www.fotonower.com/view/27805798?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/27805800?limit=200
exemples de contaminants: autre: https://www.fotonower.com/view/27805802?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/27805807?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27805408_14-10-2025_17_51_46.pdf.

Lien vers velours :https://marlene.fotonower.com/velours/27805797,27805798,27805799,27805800,27805801,27805802,27805803,27805804,27805805,27805806,27805807?tags=environnement,pet_clair,pehd,carton,background,autre,metal,pet_fonce,mal_croppe,flou,papier.


L'équipe Fotonower 202 b'' Server: nginx Date: Tue, 14 Oct 2025 15:51:50 GMT Content-Length: 0 Connection: close X-Message-Id: q3fJttFaQ4OioXTSlKSUMA Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1389435458, 1389435456, 1389435454, 1389435453, 1389435451, 1389435448, 1389435118, 1389435116, 1389435114, 1389435112, 1389435108, 1389435106, 1389434944, 1389434943, 1389434935] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435458', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435456', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435454', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435453', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435451', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435448', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435118', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435116', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435114', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435112', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435108', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435106', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389434944', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389434943', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389434935', None, None, None, None, None, '3941696') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 15 time used for this insertion : 0.036771297454833984 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.470380067825317 time spend to save output : 0.03704953193664551 total time spend for step 9 : 4.507429599761963 step10:split_time_score Tue Oct 14 17:51:50 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('16', 15),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 14102025 27805408 Nombre de photos uploadées : 15 / 23040 (0%) 14102025 27805408 Nombre de photos taguées (types de déchets): 0 / 15 (0%) 14102025 27805408 Nombre de photos taguées (volume) : 0 / 15 (0%) elapsed_time : load_data_split_time_score 1.9073486328125e-06 elapsed_time : order_list_meta_photo_and_scores 8.344650268554688e-06 ??????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0007488727569580078 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.646815299987793 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.04215036651234568 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27795759_14-10-2025_13_11_30.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27795759 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27795759 AND mptpi.`type`=3594 To do Qualite : 0.03257372249601753 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27799345_14-10-2025_15_12_47.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27799345 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27799345 AND mptpi.`type`=3594 To do Qualite : 0.10960388937114197 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27805405_14-10-2025_17_41_27.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27805405 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27805405 AND mptpi.`type`=3594 To do Qualite : 0.03357706404320987 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27805408_14-10-2025_17_51_46.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27805408 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27805408 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'14102025': {'nb_upload': 15, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1389435458, 1389435456, 1389435454, 1389435453, 1389435451, 1389435448, 1389435118, 1389435116, 1389435114, 1389435112, 1389435108, 1389435106, 1389434944, 1389434943, 1389434935] Looping around the photos to save general results len do output : 1 /27805408Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435458', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435456', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435454', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435453', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435451', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435448', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435118', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435116', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435114', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435112', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435108', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389435106', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389434944', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389434943', None, None, None, None, None, '3941696') ('3318', None, None, None, None, None, None, None, '3941696') ('3318', '27805408', '1389434935', None, None, None, None, None, '3941696') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 16 time used for this insertion : 0.037206172943115234 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.0763678550720215 time spend to save output : 0.03748607635498047 total time spend for step 10 : 2.113853931427002 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 15 set_done_treatment 39.05user 16.78system 1:28.42elapsed 63%CPU (0avgtext+0avgdata 2877748maxresident)k 1224inputs+15360outputs (3major+1472698minor)pagefaults 0swaps