python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 1385754 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['3983399'] with mtr_portfolio_ids : ['28089594'] and first list_photo_ids : [] new path : /proc/1385754/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 10 ; length of list_pids : 10 ; length of list_args : 10 time to download the photos : 1.4648935794830322 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Mon Nov 3 10:50:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10998 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-11-03 10:50:31.203469: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-11-03 10:50:31.230586: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493010000 Hz 2025-11-03 10:50:31.232519: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f2fac000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-11-03 10:50:31.232570: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-11-03 10:50:31.236493: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-11-03 10:50:31.551332: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x393d41c0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-11-03 10:50:31.551377: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-11-03 10:50:31.552872: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-11-03 10:50:31.553233: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-11-03 10:50:31.556151: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-11-03 10:50:31.558860: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-11-03 10:50:31.559326: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-11-03 10:50:31.562575: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-11-03 10:50:31.563893: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-11-03 10:50:31.567925: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-11-03 10:50:31.569347: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-11-03 10:50:31.569401: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-11-03 10:50:31.570149: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-11-03 10:50:31.570164: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-11-03 10:50:31.570172: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-11-03 10:50:31.571597: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10193 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-11-03 10:50:31.831020: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-11-03 10:50:31.831102: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-11-03 10:50:31.831121: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-11-03 10:50:31.831139: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-11-03 10:50:31.831156: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-11-03 10:50:31.831172: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-11-03 10:50:31.831188: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-11-03 10:50:31.831205: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-11-03 10:50:31.832483: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-11-03 10:50:31.833571: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-11-03 10:50:31.833599: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-11-03 10:50:31.833615: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-11-03 10:50:31.833628: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-11-03 10:50:31.833642: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-11-03 10:50:31.833658: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-11-03 10:50:31.833675: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-11-03 10:50:31.833689: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-11-03 10:50:31.834998: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-11-03 10:50:31.835026: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-11-03 10:50:31.835035: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-11-03 10:50:31.835043: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-11-03 10:50:31.836361: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10193 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-11-03 10:50:40.509058: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-11-03 10:50:40.686700: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 10 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 35.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 2 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 30.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 31.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 32.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 38.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 39.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 36.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 37.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 27.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 34.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 Detection mask done ! Trying to reset tf kernel 1386147 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5010 tf kernel not reseted sub process len(results) : 10 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 10 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 6813 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.00019216537475585938 nb_pixel_total : 3188 time to create 1 rle with old method : 0.0038945674896240234 length of segment : 56 time for calcul the mask position with numpy : 0.0004680156707763672 nb_pixel_total : 17202 time to create 1 rle with old method : 0.020588159561157227 length of segment : 175 time for calcul the mask position with numpy : 0.0003020763397216797 nb_pixel_total : 7839 time to create 1 rle with old method : 0.009301185607910156 length of segment : 177 time for calcul the mask position with numpy : 6.461143493652344e-05 nb_pixel_total : 1516 time to create 1 rle with old method : 0.0018923282623291016 length of segment : 38 time for calcul the mask position with numpy : 0.05210542678833008 nb_pixel_total : 1529504 time to create 1 rle with new method : 0.1727290153503418 length of segment : 1170 time for calcul the mask position with numpy : 0.00021004676818847656 nb_pixel_total : 9703 time to create 1 rle with old method : 0.011499404907226562 length of segment : 92 time for calcul the mask position with numpy : 0.0003380775451660156 nb_pixel_total : 18486 time to create 1 rle with old method : 0.021684646606445312 length of segment : 141 time for calcul the mask position with numpy : 0.001943826675415039 nb_pixel_total : 101446 time to create 1 rle with old method : 0.11222219467163086 length of segment : 573 time for calcul the mask position with numpy : 0.00019121170043945312 nb_pixel_total : 7019 time to create 1 rle with old method : 0.008261919021606445 length of segment : 84 time for calcul the mask position with numpy : 0.00011444091796875 nb_pixel_total : 4999 time to create 1 rle with old method : 0.005837678909301758 length of segment : 79 time for calcul the mask position with numpy : 0.00015473365783691406 nb_pixel_total : 6158 time to create 1 rle with old method : 0.007039785385131836 length of segment : 106 time for calcul the mask position with numpy : 0.0002319812774658203 nb_pixel_total : 15759 time to create 1 rle with old method : 0.01724863052368164 length of segment : 193 time for calcul the mask position with numpy : 0.02190995216369629 nb_pixel_total : 1128632 time to create 1 rle with new method : 0.6360936164855957 length of segment : 1807 time for calcul the mask position with numpy : 0.002416372299194336 nb_pixel_total : 114406 time to create 1 rle with old method : 0.1265108585357666 length of segment : 537 time for calcul the mask position with numpy : 0.00025725364685058594 nb_pixel_total : 5248 time to create 1 rle with old method : 0.006009340286254883 length of segment : 75 time for calcul the mask position with numpy : 0.0004134178161621094 nb_pixel_total : 8912 time to create 1 rle with old method : 0.010205268859863281 length of segment : 204 time for calcul the mask position with numpy : 0.000335693359375 nb_pixel_total : 10550 time to create 1 rle with old method : 0.012520074844360352 length of segment : 99 time for calcul the mask position with numpy : 0.002463817596435547 nb_pixel_total : 93138 time to create 1 rle with old method : 0.10166668891906738 length of segment : 662 time for calcul the mask position with numpy : 0.018082141876220703 nb_pixel_total : 760038 time to create 1 rle with new method : 0.05249190330505371 length of segment : 995 time for calcul the mask position with numpy : 0.0007259845733642578 nb_pixel_total : 14703 time to create 1 rle with old method : 0.016860008239746094 length of segment : 203 time for calcul the mask position with numpy : 0.0003840923309326172 nb_pixel_total : 6546 time to create 1 rle with old method : 0.00764012336730957 length of segment : 132 time spent for convertir_results : 2.9071977138519287 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 21 chid ids of type : 3594 Number RLEs to save : 7598 save missing photos in datou_result : time spend for datou_step_exec : 24.84772753715515 time spend to save output : 0.5185365676879883 total time spend for step 1 : 25.36626410484314 step2:crop_condition Mon Nov 3 10:50:53 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 21 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 3736932 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1762163454_1385754 INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1762163455), 0.0, 0.0, 14, '', 0, 0, '1762163427_1385754_1392003698_abe95462ef404751b1bea5f473dadcf5_rle_crop_4016314591_0.png', 0, 78, 55, 0, 1762163455,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1762163455), 0.0, 0.0, 14, '', 0, 0, '1762163427_1385754_1392003620_91a59dec17a20d481b8fa6e8d8a80ddc_rle_crop_4016314599_0.png', 0, 121, 82, 0, 1762163455,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1762163455), 0.0, 0.0, 14, '', 0, 0, '1762163427_1385754_1392003615_2ffffac4187a558ae327a3dce8e554ad_rle_crop_4016314606_0.png', 0, 75, 175, 0, 1762163455,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 3 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.1596627235412598 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 3736932 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1762163456_1385754 INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1762163456), 0.0, 0.0, 14, '', 0, 0, '1762163427_1385754_1392003698_abe95462ef404751b1bea5f473dadcf5_rle_crop_4016314593_0.png', 0, 122, 136, 0, 1762163456,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1762163456), 0.0, 0.0, 14, '', 0, 0, '1762163427_1385754_1392003620_91a59dec17a20d481b8fa6e8d8a80ddc_rle_crop_4016314601_0.png', 0, 118, 106, 0, 1762163456,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1762163456), 0.0, 0.0, 14, '', 0, 0, '1762163427_1385754_1392003581_27e4d22bc0a371eb03dd79c41be52967_rle_crop_4016314611_0.png', 0, 127, 132, 0, 1762163456,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 3 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.9698972702026367 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1762163457_1385754 INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1762163457), 0.0, 0.0, 14, '', 0, 0, '1762163427_1385754_1392003615_2ffffac4187a558ae327a3dce8e554ad_rle_crop_4016314607_0.png', 0, 134, 97, 0, 1762163457,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.6279497146606445 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 14 About to insert : list_path_to_insert length 14 new photo from crops ! About to upload 14 photos upload in portfolio : 3736932 init cache_photo without model_param we have 14 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1762163473_1385754 INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1762163476), 0.0, 0.0, 14, '', 0, 0, '1762163427_1385754_1392003698_abe95462ef404751b1bea5f473dadcf5_rle_crop_4016314594_0.png', 0, 48, 38, 0, 1762163476,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1762163476), 0.0, 0.0, 14, '', 0, 0, '1762163427_1385754_1392003698_abe95462ef404751b1bea5f473dadcf5_rle_crop_4016314592_0.png', 0, 127, 175, 0, 1762163476,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1762163476), 0.0, 0.0, 14, '', 0, 0, '1762163427_1385754_1392003623_d0e7f011be11dd2c6805b995055b5185_rle_crop_4016314595_0.png', 0, 1754, 971, 0, 1762163476,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1762163476), 0.0, 0.0, 14, '', 0, 0, '1762163427_1385754_1392003623_d0e7f011be11dd2c6805b995055b5185_rle_crop_4016314596_0.png', 0, 172, 79, 0, 1762163476,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1762163476), 0.0, 0.0, 14, '', 0, 0, '1762163427_1385754_1392003623_d0e7f011be11dd2c6805b995055b5185_rle_crop_4016314598_0.png', 0, 321, 569, 0, 1762163476,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1762163476), 0.0, 0.0, 14, '', 0, 0, '1762163427_1385754_1392003623_d0e7f011be11dd2c6805b995055b5185_rle_crop_4016314597_0.png', 0, 206, 140, 0, 1762163476,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1762163476), 0.0, 0.0, 14, '', 0, 0, '1762163427_1385754_1392003620_91a59dec17a20d481b8fa6e8d8a80ddc_rle_crop_4016314600_0.png', 0, 90, 79, 0, 1762163476,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1762163476), 0.0, 0.0, 14, '', 0, 0, '1762163427_1385754_1392003618_6208166ef12dabcd7e251af0b8893e4b_rle_crop_4016314603_0.png', 0, 1271, 994, 0, 1762163476,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1762163476), 0.0, 0.0, 14, '', 0, 0, '1762163427_1385754_1392003618_6208166ef12dabcd7e251af0b8893e4b_rle_crop_4016314602_0.png', 0, 131, 156, 0, 1762163476,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1762163476), 0.0, 0.0, 14, '', 0, 0, '1762163427_1385754_1392003618_6208166ef12dabcd7e251af0b8893e4b_rle_crop_4016314604_0.png', 0, 350, 531, 0, 1762163476,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1762163476), 0.0, 0.0, 14, '', 0, 0, '1762163427_1385754_1392003615_2ffffac4187a558ae327a3dce8e554ad_rle_crop_4016314605_0.png', 0, 84, 75, 0, 1762163476,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1762163476), 0.0, 0.0, 14, '', 0, 0, '1762163427_1385754_1392003610_674fb53cd716eaefe700bbe9836d25c4_rle_crop_4016314608_0.png', 0, 332, 495, 0, 1762163476,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1762163476), 0.0, 0.0, 14, '', 0, 0, '1762163427_1385754_1392003607_b47e762401e1fac0545bce8c3388327c_rle_crop_4016314609_0.png', 0, 1058, 988, 0, 1762163476,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1762163476), 0.0, 0.0, 14, '', 0, 0, '1762163427_1385754_1392003581_27e4d22bc0a371eb03dd79c41be52967_rle_crop_4016314610_0.png', 0, 109, 189, 0, 1762163476,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 14 photos in the portfolio 3736932 time of upload the photos Elapsed time : 3.8444957733154297 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1392003702, 1392003698, 1392003623, 1392003620, 1392003618, 1392003615, 1392003610, 1392003607, 1392003581, 1392003577] Looping around the photos to save general results len do output : 21 /1392018516Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1392018517Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1392018518Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1392018519Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1392018520Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1392018521Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1392018522Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1392018537Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1392018538Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1392018539Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1392018540Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1392018541Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1392018542Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1392018543Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1392018544Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1392018545Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1392018546Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1392018547Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1392018548Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1392018550Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1392018551Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003702', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003698', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003623', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003620', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003618', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003615', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003610', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003607', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003581', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003577', None, None, None, None, None, '3983399') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 73 time used for this insertion : 0.019459009170532227 save_final save missing photos in datou_result : time spend for datou_step_exec : 23.537535667419434 time spend to save output : 0.020349740982055664 total time spend for step 2 : 23.55788540840149 step3:rle_unique_nms_with_priority Mon Nov 3 10:51:17 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 21 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++ No data in photo_id : 1392003702 nb_obj : 4 nb_hashtags : 3 time to prepare the origin masks : 0.8315656185150146 time for calcul the mask position with numpy : 0.173661470413208 nb_pixel_total : 2043855 time to create 1 rle with new method : 0.2587919235229492 time for calcul the mask position with numpy : 0.006505727767944336 nb_pixel_total : 1516 time to create 1 rle with old method : 0.0017380714416503906 time for calcul the mask position with numpy : 0.006465435028076172 nb_pixel_total : 7839 time to create 1 rle with old method : 0.009000301361083984 time for calcul the mask position with numpy : 0.006613969802856445 nb_pixel_total : 17202 time to create 1 rle with old method : 0.01909184455871582 time for calcul the mask position with numpy : 0.006491899490356445 nb_pixel_total : 3188 time to create 1 rle with old method : 0.0035448074340820312 create new chi : 0.5018162727355957 time to delete rle : 0.02659893035888672 batch 1 Loaded 9 chid ids of type : 3594 +++++Number RLEs to save : 1972 TO DO : save crop sub photo not yet done ! save time : 0.1726210117340088 nb_obj : 4 nb_hashtags : 1 time to prepare the origin masks : 0.17605829238891602 time for calcul the mask position with numpy : 0.011327981948852539 nb_pixel_total : 416216 time to create 1 rle with new method : 0.1027519702911377 time for calcul the mask position with numpy : 0.008538007736206055 nb_pixel_total : 101446 time to create 1 rle with old method : 0.11259889602661133 time for calcul the mask position with numpy : 0.0069048404693603516 nb_pixel_total : 16867 time to create 1 rle with old method : 0.027292490005493164 time for calcul the mask position with numpy : 0.0074536800384521484 nb_pixel_total : 9567 time to create 1 rle with old method : 0.010887861251831055 time for calcul the mask position with numpy : 0.018271446228027344 nb_pixel_total : 1529504 time to create 1 rle with new method : 0.05363917350769043 create new chi : 0.3605525493621826 time to delete rle : 0.0006079673767089844 batch 1 Loaded 9 chid ids of type : 3594 ++++++++Number RLEs to save : 4971 TO DO : save crop sub photo not yet done ! save time : 0.3360595703125 nb_obj : 3 nb_hashtags : 3 time to prepare the origin masks : 0.05805325508117676 time for calcul the mask position with numpy : 0.1201930046081543 nb_pixel_total : 2055424 time to create 1 rle with new method : 0.10708427429199219 time for calcul the mask position with numpy : 0.006899595260620117 nb_pixel_total : 6158 time to create 1 rle with old method : 0.006820201873779297 time for calcul the mask position with numpy : 0.012012004852294922 nb_pixel_total : 4999 time to create 1 rle with old method : 0.005718708038330078 time for calcul the mask position with numpy : 0.013807296752929688 nb_pixel_total : 7019 time to create 1 rle with old method : 0.009038209915161133 create new chi : 0.29492807388305664 time to delete rle : 0.00038886070251464844 batch 1 Loaded 7 chid ids of type : 3594 +++Number RLEs to save : 1618 TO DO : save crop sub photo not yet done ! save time : 0.1552746295928955 nb_obj : 3 nb_hashtags : 1 time to prepare the origin masks : 0.13228893280029297 time for calcul the mask position with numpy : 0.017499923706054688 nb_pixel_total : 814803 time to create 1 rle with new method : 0.19906210899353027 time for calcul the mask position with numpy : 0.008139371871948242 nb_pixel_total : 114406 time to create 1 rle with old method : 0.15101909637451172 time for calcul the mask position with numpy : 0.03815460205078125 nb_pixel_total : 1128632 time to create 1 rle with new method : 0.1408674716949463 time for calcul the mask position with numpy : 0.006548166275024414 nb_pixel_total : 15759 time to create 1 rle with old method : 0.017902612686157227 create new chi : 0.5943522453308105 time to delete rle : 0.0008513927459716797 batch 1 Loaded 7 chid ids of type : 3594 +++++Number RLEs to save : 6154 TO DO : save crop sub photo not yet done ! save time : 0.41889214515686035 nb_obj : 3 nb_hashtags : 3 time to prepare the origin masks : 0.15840411186218262 time for calcul the mask position with numpy : 0.06458139419555664 nb_pixel_total : 2048890 time to create 1 rle with new method : 0.1093442440032959 time for calcul the mask position with numpy : 0.007822751998901367 nb_pixel_total : 10550 time to create 1 rle with old method : 0.012637615203857422 time for calcul the mask position with numpy : 0.006494998931884766 nb_pixel_total : 8912 time to create 1 rle with old method : 0.012837409973144531 time for calcul the mask position with numpy : 0.008720874786376953 nb_pixel_total : 5248 time to create 1 rle with old method : 0.009881019592285156 create new chi : 0.24348115921020508 time to delete rle : 0.0006084442138671875 batch 1 Loaded 7 chid ids of type : 3594 +++++Number RLEs to save : 1836 TO DO : save crop sub photo not yet done ! save time : 0.16050314903259277 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.04649043083190918 time for calcul the mask position with numpy : 0.02355813980102539 nb_pixel_total : 1980462 time to create 1 rle with new method : 0.13288021087646484 time for calcul the mask position with numpy : 0.011661052703857422 nb_pixel_total : 93138 time to create 1 rle with old method : 0.10864782333374023 create new chi : 0.2855098247528076 time to delete rle : 0.0004818439483642578 batch 1 Loaded 3 chid ids of type : 3594 ++++++Number RLEs to save : 2404 TO DO : save crop sub photo not yet done ! save time : 0.18827271461486816 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.17458462715148926 time for calcul the mask position with numpy : 0.04674220085144043 nb_pixel_total : 1313562 time to create 1 rle with new method : 0.10680055618286133 time for calcul the mask position with numpy : 0.01583075523376465 nb_pixel_total : 760038 time to create 1 rle with new method : 0.08179926872253418 create new chi : 0.2643930912017822 time to delete rle : 0.00036597251892089844 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 3070 TO DO : save crop sub photo not yet done ! save time : 0.242659330368042 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.29085731506347656 time for calcul the mask position with numpy : 0.1715562343597412 nb_pixel_total : 2052351 time to create 1 rle with new method : 0.11619400978088379 time for calcul the mask position with numpy : 0.00632166862487793 nb_pixel_total : 6546 time to create 1 rle with old method : 0.007430315017700195 time for calcul the mask position with numpy : 0.00605010986328125 nb_pixel_total : 14703 time to create 1 rle with old method : 0.016452312469482422 create new chi : 0.3344883918762207 time to delete rle : 0.00040602684020996094 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 1750 TO DO : save crop sub photo not yet done ! save time : 0.15067768096923828 No data in photo_id : 1392003577 map_output_result : {1392003702: (0.0, 'Should be the crop_list due to order', 0.0), 1392003698: (0.0, 'Should be the crop_list due to order', 0), 1392003623: (0.0, 'Should be the crop_list due to order', 0), 1392003620: (0.0, 'Should be the crop_list due to order', 0), 1392003618: (0.0, 'Should be the crop_list due to order', 0), 1392003615: (0.0, 'Should be the crop_list due to order', 0), 1392003610: (0.0, 'Should be the crop_list due to order', 0), 1392003607: (0.0, 'Should be the crop_list due to order', 0), 1392003581: (0.0, 'Should be the crop_list due to order', 0), 1392003577: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1392003702, 1392003698, 1392003623, 1392003620, 1392003618, 1392003615, 1392003610, 1392003607, 1392003581, 1392003577] Looping around the photos to save general results len do output : 10 /1392003702.Didn't retrieve data . /1392003698.Didn't retrieve data . /1392003623.Didn't retrieve data . /1392003620.Didn't retrieve data . /1392003618.Didn't retrieve data . /1392003615.Didn't retrieve data . /1392003610.Didn't retrieve data . /1392003607.Didn't retrieve data . /1392003581.Didn't retrieve data . /1392003577.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003702', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003698', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003623', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003620', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003618', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003615', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003610', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003607', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003581', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003577', None, None, None, None, None, '3983399') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.01852869987487793 save_final save missing photos in datou_result : time spend for datou_step_exec : 6.9333815574646 time spend to save output : 0.01896834373474121 total time spend for step 3 : 6.952349901199341 step4:ventilate_hashtags_in_portfolio Mon Nov 3 10:51:24 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 28089594 get user id for portfolio 28089594 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=28089594 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','autre','carton','mal_croppe','background','metal','pehd','environnement','pet_clair','flou','pet_fonce')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=28089594 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','autre','carton','mal_croppe','background','metal','pehd','environnement','pet_clair','flou','pet_fonce')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=28089594 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','autre','carton','mal_croppe','background','metal','pehd','environnement','pet_clair','flou','pet_fonce')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://marlene.fotonower.com/velours/28090152,28090153,28090154,28090155,28090156,28090157,28090158,28090159,28090160,28090161,28090162?tags=papier,autre,carton,mal_croppe,background,metal,pehd,environnement,pet_clair,flou,pet_fonce Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1392003702, 1392003698, 1392003623, 1392003620, 1392003618, 1392003615, 1392003610, 1392003607, 1392003581, 1392003577] Looping around the photos to save general results len do output : 1 /28089594. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003702', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003698', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003623', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003620', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003618', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003615', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003610', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003607', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003581', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003577', None, None, None, None, None, '3983399') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.017386198043823242 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.7616791725158691 time spend to save output : 0.01766180992126465 total time spend for step 4 : 1.7793409824371338 step5:final Mon Nov 3 10:51:26 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1392003702: ('0.1863057966820988',), 1392003698: ('0.1863057966820988',), 1392003623: ('0.1863057966820988',), 1392003620: ('0.1863057966820988',), 1392003618: ('0.1863057966820988',), 1392003615: ('0.1863057966820988',), 1392003610: ('0.1863057966820988',), 1392003607: ('0.1863057966820988',), 1392003581: ('0.1863057966820988',), 1392003577: ('0.1863057966820988',)} new output for save of step final : {1392003702: ('0.1863057966820988',), 1392003698: ('0.1863057966820988',), 1392003623: ('0.1863057966820988',), 1392003620: ('0.1863057966820988',), 1392003618: ('0.1863057966820988',), 1392003615: ('0.1863057966820988',), 1392003610: ('0.1863057966820988',), 1392003607: ('0.1863057966820988',), 1392003581: ('0.1863057966820988',), 1392003577: ('0.1863057966820988',)} [1392003702, 1392003698, 1392003623, 1392003620, 1392003618, 1392003615, 1392003610, 1392003607, 1392003581, 1392003577] Looping around the photos to save general results len do output : 10 /1392003702.Didn't retrieve data . /1392003698.Didn't retrieve data . /1392003623.Didn't retrieve data . /1392003620.Didn't retrieve data . /1392003618.Didn't retrieve data . /1392003615.Didn't retrieve data . /1392003610.Didn't retrieve data . /1392003607.Didn't retrieve data . /1392003581.Didn't retrieve data . /1392003577.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003702', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003698', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003623', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003620', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003618', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003615', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003610', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003607', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003581', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003577', None, None, None, None, None, '3983399') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.01651310920715332 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.12722396850585938 time spend to save output : 0.016899824142456055 total time spend for step 5 : 0.14412379264831543 step6:blur_detection Mon Nov 3 10:51:26 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1762163427_1385754_1392003702_e2c5b27be32d70ef3580f8953d547c32.jpg resize: (1080, 1920) 1392003702 -2.149102010067177 treat image : temp/1762163427_1385754_1392003698_abe95462ef404751b1bea5f473dadcf5.jpg resize: (1080, 1920) 1392003698 -2.6624983013475894 treat image : temp/1762163427_1385754_1392003623_d0e7f011be11dd2c6805b995055b5185.jpg resize: (1080, 1920) 1392003623 -2.1087451183099724 treat image : temp/1762163427_1385754_1392003620_91a59dec17a20d481b8fa6e8d8a80ddc.jpg resize: (1080, 1920) 1392003620 -1.9012959772020763 treat image : temp/1762163427_1385754_1392003618_6208166ef12dabcd7e251af0b8893e4b.jpg resize: (1080, 1920) 1392003618 -2.308576226593558 treat image : temp/1762163427_1385754_1392003615_2ffffac4187a558ae327a3dce8e554ad.jpg resize: (1080, 1920) 1392003615 -2.3179092149295264 treat image : temp/1762163427_1385754_1392003610_674fb53cd716eaefe700bbe9836d25c4.jpg resize: (1080, 1920) 1392003610 -2.1859429998720654 treat image : temp/1762163427_1385754_1392003607_b47e762401e1fac0545bce8c3388327c.jpg resize: (1080, 1920) 1392003607 -2.7198144220010754 treat image : temp/1762163427_1385754_1392003581_27e4d22bc0a371eb03dd79c41be52967.jpg resize: (1080, 1920) 1392003581 -1.9845362560089899 treat image : temp/1762163427_1385754_1392003577_8496a5bad9e938f6faea3d4e1efdd09a.jpg resize: (1080, 1920) 1392003577 -1.218495311560425 treat image : temp/1762163427_1385754_1392003698_abe95462ef404751b1bea5f473dadcf5_rle_crop_4016314591_0.png resize: (55, 78) 1392018516 -0.080860546206311 treat image : temp/1762163427_1385754_1392003620_91a59dec17a20d481b8fa6e8d8a80ddc_rle_crop_4016314599_0.png resize: (82, 121) 1392018517 -0.7960116342895526 treat image : temp/1762163427_1385754_1392003615_2ffffac4187a558ae327a3dce8e554ad_rle_crop_4016314606_0.png resize: (175, 75) 1392018518 -2.199958172292437 treat image : temp/1762163427_1385754_1392003698_abe95462ef404751b1bea5f473dadcf5_rle_crop_4016314593_0.png resize: (136, 122) 1392018519 -2.249737522794793 treat image : temp/1762163427_1385754_1392003620_91a59dec17a20d481b8fa6e8d8a80ddc_rle_crop_4016314601_0.png resize: (106, 118) 1392018520 -2.0720457789596995 treat image : temp/1762163427_1385754_1392003581_27e4d22bc0a371eb03dd79c41be52967_rle_crop_4016314611_0.png resize: (132, 127) 1392018521 -2.2506649972722093 treat image : temp/1762163427_1385754_1392003615_2ffffac4187a558ae327a3dce8e554ad_rle_crop_4016314607_0.png resize: (97, 134) 1392018522 -3.878019538315774 treat image : temp/1762163427_1385754_1392003698_abe95462ef404751b1bea5f473dadcf5_rle_crop_4016314594_0.png resize: (38, 48) 1392018537 0.043974782144010724 treat image : temp/1762163427_1385754_1392003698_abe95462ef404751b1bea5f473dadcf5_rle_crop_4016314592_0.png resize: (175, 127) 1392018538 -0.9241183684772221 treat image : temp/1762163427_1385754_1392003623_d0e7f011be11dd2c6805b995055b5185_rle_crop_4016314595_0.png resize: (971, 1754) 1392018539 -2.4511123257863114 treat image : temp/1762163427_1385754_1392003623_d0e7f011be11dd2c6805b995055b5185_rle_crop_4016314596_0.png resize: (79, 172) 1392018540 -2.832470341973851 treat image : temp/1762163427_1385754_1392003623_d0e7f011be11dd2c6805b995055b5185_rle_crop_4016314598_0.png resize: (569, 321) 1392018541 -0.0755845814949007 treat image : temp/1762163427_1385754_1392003623_d0e7f011be11dd2c6805b995055b5185_rle_crop_4016314597_0.png resize: (140, 206) 1392018542 -1.7972651108733084 treat image : temp/1762163427_1385754_1392003620_91a59dec17a20d481b8fa6e8d8a80ddc_rle_crop_4016314600_0.png resize: (79, 90) 1392018543 -4.06568258317375 treat image : temp/1762163427_1385754_1392003618_6208166ef12dabcd7e251af0b8893e4b_rle_crop_4016314603_0.png resize: (994, 1271) 1392018544 -2.5190371767224002 treat image : temp/1762163427_1385754_1392003618_6208166ef12dabcd7e251af0b8893e4b_rle_crop_4016314602_0.png resize: (156, 131) 1392018545 -3.3347890105788687 treat image : temp/1762163427_1385754_1392003618_6208166ef12dabcd7e251af0b8893e4b_rle_crop_4016314604_0.png resize: (531, 350) 1392018546 0.16832326020890048 treat image : temp/1762163427_1385754_1392003615_2ffffac4187a558ae327a3dce8e554ad_rle_crop_4016314605_0.png resize: (75, 84) 1392018547 -2.609652206504572 treat image : temp/1762163427_1385754_1392003610_674fb53cd716eaefe700bbe9836d25c4_rle_crop_4016314608_0.png resize: (495, 332) 1392018548 -1.409131984667061 treat image : temp/1762163427_1385754_1392003607_b47e762401e1fac0545bce8c3388327c_rle_crop_4016314609_0.png resize: (988, 1058) 1392018550 -0.25794598562222576 treat image : temp/1762163427_1385754_1392003581_27e4d22bc0a371eb03dd79c41be52967_rle_crop_4016314610_0.png resize: (189, 109) 1392018551 -1.573234306477011 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 31 time used for this insertion : 0.01505732536315918 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 31 time used for this insertion : 0.014863729476928711 save missing photos in datou_result : time spend for datou_step_exec : 10.169604539871216 time spend to save output : 0.035166263580322266 total time spend for step 6 : 10.204770803451538 step7:brightness Mon Nov 3 10:51:36 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1762163427_1385754_1392003702_e2c5b27be32d70ef3580f8953d547c32.jpg treat image : temp/1762163427_1385754_1392003698_abe95462ef404751b1bea5f473dadcf5.jpg treat image : temp/1762163427_1385754_1392003623_d0e7f011be11dd2c6805b995055b5185.jpg treat image : temp/1762163427_1385754_1392003620_91a59dec17a20d481b8fa6e8d8a80ddc.jpg treat image : temp/1762163427_1385754_1392003618_6208166ef12dabcd7e251af0b8893e4b.jpg treat image : temp/1762163427_1385754_1392003615_2ffffac4187a558ae327a3dce8e554ad.jpg treat image : temp/1762163427_1385754_1392003610_674fb53cd716eaefe700bbe9836d25c4.jpg treat image : temp/1762163427_1385754_1392003607_b47e762401e1fac0545bce8c3388327c.jpg treat image : temp/1762163427_1385754_1392003581_27e4d22bc0a371eb03dd79c41be52967.jpg treat image : temp/1762163427_1385754_1392003577_8496a5bad9e938f6faea3d4e1efdd09a.jpg treat image : temp/1762163427_1385754_1392003698_abe95462ef404751b1bea5f473dadcf5_rle_crop_4016314591_0.png treat image : temp/1762163427_1385754_1392003620_91a59dec17a20d481b8fa6e8d8a80ddc_rle_crop_4016314599_0.png treat image : temp/1762163427_1385754_1392003615_2ffffac4187a558ae327a3dce8e554ad_rle_crop_4016314606_0.png treat image : temp/1762163427_1385754_1392003698_abe95462ef404751b1bea5f473dadcf5_rle_crop_4016314593_0.png treat image : temp/1762163427_1385754_1392003620_91a59dec17a20d481b8fa6e8d8a80ddc_rle_crop_4016314601_0.png treat image : temp/1762163427_1385754_1392003581_27e4d22bc0a371eb03dd79c41be52967_rle_crop_4016314611_0.png treat image : temp/1762163427_1385754_1392003615_2ffffac4187a558ae327a3dce8e554ad_rle_crop_4016314607_0.png treat image : temp/1762163427_1385754_1392003698_abe95462ef404751b1bea5f473dadcf5_rle_crop_4016314594_0.png treat image : temp/1762163427_1385754_1392003698_abe95462ef404751b1bea5f473dadcf5_rle_crop_4016314592_0.png treat image : temp/1762163427_1385754_1392003623_d0e7f011be11dd2c6805b995055b5185_rle_crop_4016314595_0.png treat image : temp/1762163427_1385754_1392003623_d0e7f011be11dd2c6805b995055b5185_rle_crop_4016314596_0.png treat image : temp/1762163427_1385754_1392003623_d0e7f011be11dd2c6805b995055b5185_rle_crop_4016314598_0.png treat image : temp/1762163427_1385754_1392003623_d0e7f011be11dd2c6805b995055b5185_rle_crop_4016314597_0.png treat image : temp/1762163427_1385754_1392003620_91a59dec17a20d481b8fa6e8d8a80ddc_rle_crop_4016314600_0.png treat image : temp/1762163427_1385754_1392003618_6208166ef12dabcd7e251af0b8893e4b_rle_crop_4016314603_0.png treat image : temp/1762163427_1385754_1392003618_6208166ef12dabcd7e251af0b8893e4b_rle_crop_4016314602_0.png treat image : temp/1762163427_1385754_1392003618_6208166ef12dabcd7e251af0b8893e4b_rle_crop_4016314604_0.png treat image : temp/1762163427_1385754_1392003615_2ffffac4187a558ae327a3dce8e554ad_rle_crop_4016314605_0.png treat image : temp/1762163427_1385754_1392003610_674fb53cd716eaefe700bbe9836d25c4_rle_crop_4016314608_0.png treat image : temp/1762163427_1385754_1392003607_b47e762401e1fac0545bce8c3388327c_rle_crop_4016314609_0.png treat image : temp/1762163427_1385754_1392003581_27e4d22bc0a371eb03dd79c41be52967_rle_crop_4016314610_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 31 time used for this insertion : 0.015820741653442383 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 31 time used for this insertion : 0.01714611053466797 save missing photos in datou_result : time spend for datou_step_exec : 2.830142021179199 time spend to save output : 0.03730201721191406 total time spend for step 7 : 2.8674440383911133 step8:velours_tree Mon Nov 3 10:51:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.13771486282348633 time spend to save output : 4.5299530029296875e-05 total time spend for step 8 : 0.13776016235351562 step9:send_mail_cod Mon Nov 3 10:51:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P28089594_03-11-2025_10_51_39.pdf 28090152 change filename to text .change filename to text .change filename to text .imagette280901521762163499 28090153 imagette280901531762163499 28090154 change filename to text .change filename to text .change filename to text .imagette280901541762163499 28090155 imagette280901551762163500 28090156 imagette280901561762163500 28090157 change filename to text .imagette280901571762163500 28090158 imagette280901581762163500 28090160 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette280901601762163500 28090161 imagette280901611762163501 28090162 imagette280901621762163501 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=28089594 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://marlene.fotonower.com/velours/28090152,28090153,28090154,28090155,28090156,28090157,28090158,28090159,28090160,28090161,28090162?tags=papier,autre,carton,mal_croppe,background,metal,pehd,environnement,pet_clair,flou,pet_fonce args[1392003702] : ((1392003702, -2.149102010067177, 492609224), (1392003702, 0.6659018397465792, 2107752395), '0.1863057966820988') We are sending mail with results at report@fotonower.com args[1392003698] : ((1392003698, -2.6624983013475894, 492609224), (1392003698, 0.6368648111514714, 2107752395), '0.1863057966820988') We are sending mail with results at report@fotonower.com args[1392003623] : ((1392003623, -2.1087451183099724, 492609224), (1392003623, 0.509779545853025, 2107752395), '0.1863057966820988') We are sending mail with results at report@fotonower.com args[1392003620] : ((1392003620, -1.9012959772020763, 492688767), (1392003620, 0.4990763750335618, 2107752395), '0.1863057966820988') We are sending mail with results at report@fotonower.com args[1392003618] : ((1392003618, -2.308576226593558, 492609224), (1392003618, 0.5271996432297957, 2107752395), '0.1863057966820988') We are sending mail with results at report@fotonower.com args[1392003615] : ((1392003615, -2.3179092149295264, 492609224), (1392003615, 0.7106978723590713, 2107752395), '0.1863057966820988') We are sending mail with results at report@fotonower.com args[1392003610] : ((1392003610, -2.1859429998720654, 492609224), (1392003610, 0.5917775721155694, 2107752395), '0.1863057966820988') We are sending mail with results at report@fotonower.com args[1392003607] : ((1392003607, -2.7198144220010754, 492609224), (1392003607, 0.7071257722130332, 2107752395), '0.1863057966820988') We are sending mail with results at report@fotonower.com args[1392003581] : ((1392003581, -1.9845362560089899, 492688767), (1392003581, 0.540897046079805, 2107752395), '0.1863057966820988') We are sending mail with results at report@fotonower.com args[1392003577] : ((1392003577, -1.218495311560425, 492688767), (1392003577, 0.5532009582246199, 2107752395), '0.1863057966820988') We are sending mail with results at report@fotonower.com refus_total : 0.1863057966820988 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos_view ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=28089594 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P28089594_03-11-2025_10_51_39.pdf results_Auto_P28089594_03-11-2025_10_51_39.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P28089594_03-11-2025_10_51_39.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','28089594','results_Auto_P28089594_03-11-2025_10_51_39.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P28089594_03-11-2025_10_51_39.pdf','pdf','','0.48','0.1863057966820988') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/28089594

https://www.fotonower.com/image?json=false&list_photos_id=1392003702
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1392003698
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1392003623
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1392003620
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1392003618
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1392003615
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1392003610
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1392003607
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1392003581
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1392003577
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 18.63%
Veuillez trouver les photos des contaminants.

exemples de contaminants: papier: https://www.fotonower.com/view/28090152?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/28090154?limit=200
exemples de contaminants: metal: https://www.fotonower.com/view/28090157?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/28090160?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P28089594_03-11-2025_10_51_39.pdf.

Lien vers velours :https://marlene.fotonower.com/velours/28090152,28090153,28090154,28090155,28090156,28090157,28090158,28090159,28090160,28090161,28090162?tags=papier,autre,carton,mal_croppe,background,metal,pehd,environnement,pet_clair,flou,pet_fonce.


L'équipe Fotonower 202 b'' Server: nginx Date: Mon, 03 Nov 2025 09:51:43 GMT Content-Length: 0 Connection: close X-Message-Id: urN5sViRRgu9kjaFpWSpDA Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1392003702, 1392003698, 1392003623, 1392003620, 1392003618, 1392003615, 1392003610, 1392003607, 1392003581, 1392003577] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003702', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003698', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003623', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003620', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003618', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003615', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003610', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003607', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003581', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003577', None, None, None, None, None, '3983399') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.017827749252319336 save_final save missing photos in datou_result : time spend for datou_step_exec : 3.7615416049957275 time spend to save output : 0.018044471740722656 total time spend for step 9 : 3.77958607673645 step10:split_time_score Mon Nov 3 10:51:43 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('08', 10),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 03112025 28089594 Nombre de photos uploadées : 10 / 23040 (0%) 03112025 28089594 Nombre de photos taguées (types de déchets): 0 / 10 (0%) 03112025 28089594 Nombre de photos taguées (volume) : 0 / 10 (0%) elapsed_time : load_data_split_time_score 1.430511474609375e-06 elapsed_time : order_list_meta_photo_and_scores 3.5762786865234375e-06 ?????????? elapsed_time : fill_and_build_computed_from_old_data 0.00043964385986328125 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.21233153343200684 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 28089591 order by id desc limit 1 Qualite : 0.1863057966820988 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P28089594_03-11-2025_10_51_39.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 28089594 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=28089594 AND mptpi.`type`=3594 To do Qualite : 0.11127062971536356 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P28089596_03-11-2025_10_42_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 28089596 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=28089596 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'03112025': {'nb_upload': 10, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1392003702, 1392003698, 1392003623, 1392003620, 1392003618, 1392003615, 1392003610, 1392003607, 1392003581, 1392003577] Looping around the photos to save general results len do output : 1 /28089594Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003702', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003698', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003623', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003620', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003618', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003615', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003610', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003607', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003581', None, None, None, None, None, '3983399') ('3318', None, None, None, None, None, None, None, '3983399') ('3318', '28089594', '1392003577', None, None, None, None, None, '3983399') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.015060663223266602 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.6615002155303955 time spend to save output : 0.015296697616577148 total time spend for step 10 : 1.6767969131469727 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 10 set_done_treatment 48.23user 18.94system 1:20.12elapsed 83%CPU (0avgtext+0avgdata 2699576maxresident)k 469080inputs+34520outputs (17major+1608918minor)pagefaults 0swaps