python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 991460 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['3954038'] with mtr_portfolio_ids : ['27877917'] and first list_photo_ids : [] new path : /proc/991460/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 8 ; length of list_pids : 8 ; length of list_args : 8 time to download the photos : 1.6008117198944092 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Fri Oct 17 13:00:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10998 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-10-17 13:00:35.524949: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-10-17 13:00:35.550421: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493010000 Hz 2025-10-17 13:00:35.552642: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f5998000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-10-17 13:00:35.552704: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-10-17 13:00:35.556698: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-10-17 13:00:35.847490: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x3b6e6040 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-10-17 13:00:35.847547: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-10-17 13:00:35.848872: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-10-17 13:00:35.849330: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-10-17 13:00:35.852666: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-10-17 13:00:35.855679: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-10-17 13:00:35.856205: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-10-17 13:00:35.859402: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-10-17 13:00:35.860401: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-10-17 13:00:35.868675: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-10-17 13:00:35.870669: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-10-17 13:00:35.870757: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-10-17 13:00:35.871792: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-10-17 13:00:35.871812: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-10-17 13:00:35.871825: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-10-17 13:00:35.873681: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10193 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-10-17 13:00:36.263463: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-10-17 13:00:36.263558: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-10-17 13:00:36.263583: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-10-17 13:00:36.263602: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-10-17 13:00:36.263618: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-10-17 13:00:36.263632: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-10-17 13:00:36.263646: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-10-17 13:00:36.263661: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-10-17 13:00:36.264948: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-10-17 13:00:36.266016: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-10-17 13:00:36.266043: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-10-17 13:00:36.266056: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-10-17 13:00:36.266069: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-10-17 13:00:36.266082: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-10-17 13:00:36.266094: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-10-17 13:00:36.266107: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-10-17 13:00:36.266119: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-10-17 13:00:36.267422: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-10-17 13:00:36.267449: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-10-17 13:00:36.267457: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-10-17 13:00:36.267465: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-10-17 13:00:36.268747: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10193 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-10-17 13:00:42.745350: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-10-17 13:00:42.929855: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 8 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 29.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 32.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 37.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 30.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 16.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 38.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 33.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 30.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 Detection mask done ! Trying to reset tf kernel 992190 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 4907 tf kernel not reseted sub process len(results) : 8 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 8 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10196 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0001633167266845703 nb_pixel_total : 2679 time to create 1 rle with old method : 0.0036988258361816406 length of segment : 40 time for calcul the mask position with numpy : 0.007940053939819336 nb_pixel_total : 346277 time to create 1 rle with new method : 0.04880642890930176 length of segment : 1514 time for calcul the mask position with numpy : 0.0012707710266113281 nb_pixel_total : 9912 time to create 1 rle with old method : 0.012939453125 length of segment : 304 time for calcul the mask position with numpy : 0.00013017654418945312 nb_pixel_total : 5179 time to create 1 rle with old method : 0.006143808364868164 length of segment : 99 time for calcul the mask position with numpy : 0.00015115737915039062 nb_pixel_total : 10655 time to create 1 rle with old method : 0.012092113494873047 length of segment : 158 time for calcul the mask position with numpy : 0.009984970092773438 nb_pixel_total : 739451 time to create 1 rle with new method : 0.0452885627746582 length of segment : 989 time for calcul the mask position with numpy : 0.00011205673217773438 nb_pixel_total : 3675 time to create 1 rle with old method : 0.004347562789916992 length of segment : 97 time for calcul the mask position with numpy : 0.00018215179443359375 nb_pixel_total : 8980 time to create 1 rle with old method : 0.010056495666503906 length of segment : 133 time for calcul the mask position with numpy : 0.00011301040649414062 nb_pixel_total : 6544 time to create 1 rle with old method : 0.007529735565185547 length of segment : 72 time for calcul the mask position with numpy : 0.0012521743774414062 nb_pixel_total : 99883 time to create 1 rle with old method : 0.11050534248352051 length of segment : 501 time for calcul the mask position with numpy : 0.009160041809082031 nb_pixel_total : 664335 time to create 1 rle with new method : 0.015818357467651367 length of segment : 961 time for calcul the mask position with numpy : 0.0002808570861816406 nb_pixel_total : 15090 time to create 1 rle with old method : 0.017627477645874023 length of segment : 141 time for calcul the mask position with numpy : 0.0001552104949951172 nb_pixel_total : 8231 time to create 1 rle with old method : 0.00959014892578125 length of segment : 115 time for calcul the mask position with numpy : 0.00015234947204589844 nb_pixel_total : 9518 time to create 1 rle with old method : 0.011020660400390625 length of segment : 116 time for calcul the mask position with numpy : 9.131431579589844e-05 nb_pixel_total : 3895 time to create 1 rle with old method : 0.007192134857177734 length of segment : 72 time for calcul the mask position with numpy : 0.0007588863372802734 nb_pixel_total : 24616 time to create 1 rle with old method : 0.02780604362487793 length of segment : 224 time for calcul the mask position with numpy : 0.0002586841583251953 nb_pixel_total : 7465 time to create 1 rle with old method : 0.008869409561157227 length of segment : 88 time for calcul the mask position with numpy : 0.0006604194641113281 nb_pixel_total : 28931 time to create 1 rle with old method : 0.033570051193237305 length of segment : 242 time for calcul the mask position with numpy : 0.0003514289855957031 nb_pixel_total : 11818 time to create 1 rle with old method : 0.013833761215209961 length of segment : 187 time for calcul the mask position with numpy : 0.0002429485321044922 nb_pixel_total : 5826 time to create 1 rle with old method : 0.006723880767822266 length of segment : 123 time for calcul the mask position with numpy : 0.0003371238708496094 nb_pixel_total : 8934 time to create 1 rle with old method : 0.010358810424804688 length of segment : 158 time for calcul the mask position with numpy : 0.014032125473022461 nb_pixel_total : 740783 time to create 1 rle with new method : 0.027384519577026367 length of segment : 1466 time spent for convertir_results : 2.2001359462738037 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 22 chid ids of type : 3594 Number RLEs to save : 7800 save missing photos in datou_result : time spend for datou_step_exec : 19.800962924957275 time spend to save output : 0.9796719551086426 total time spend for step 1 : 20.780634880065918 step2:crop_condition Fri Oct 17 13:00:53 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 8 ! batch 1 Loaded 22 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! About to upload 6 photos upload in portfolio : 3736932 init cache_photo without model_param we have 6 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1760698854_991460 INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760698856), 0.0, 0.0, 14, '', 0, 0, '1760698831_991460_1389741311_b4f8306dc65c3bdb7ee7c5acc8966e42_rle_crop_4000359723_0.png', 0, 93, 40, 0, 1760698856,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760698856), 0.0, 0.0, 14, '', 0, 0, '1760698831_991460_1389741301_ac3a1eaca2c726ebf3d3f70772f36fd2_rle_crop_4000359730_0.png', 0, 135, 121, 0, 1760698856,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760698856), 0.0, 0.0, 14, '', 0, 0, '1760698831_991460_1389741301_ac3a1eaca2c726ebf3d3f70772f36fd2_rle_crop_4000359726_0.png', 0, 77, 99, 0, 1760698856,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760698856), 0.0, 0.0, 14, '', 0, 0, '1760698831_991460_1389741290_ad821ed2fb15b8fbdf2216f83c832aba_rle_crop_4000359735_0.png', 0, 106, 112, 0, 1760698856,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760698856), 0.0, 0.0, 14, '', 0, 0, '1760698831_991460_1389740938_52b53bba96c4d06f0b3f5eed96049280_rle_crop_4000359739_0.png', 0, 115, 88, 0, 1760698856,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760698856), 0.0, 0.0, 14, '', 0, 0, '1760698831_991460_1389740931_a146f987bca174f4356c3dbf6b6ed245_rle_crop_4000359742_0.png', 0, 67, 123, 0, 1760698856,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 6 photos in the portfolio 3736932 time of upload the photos Elapsed time : 2.6570022106170654 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! About to upload 7 photos upload in portfolio : 3736932 init cache_photo without model_param we have 7 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1760698861_991460 INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760698862), 0.0, 0.0, 14, '', 0, 0, '1760698831_991460_1389741311_b4f8306dc65c3bdb7ee7c5acc8966e42_rle_crop_4000359724_0.png', 0, 876, 887, 0, 1760698862,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760698862), 0.0, 0.0, 14, '', 0, 0, '1760698831_991460_1389741310_b6565a6d046262f99bd2fe5136643b79_rle_crop_4000359725_0.png', 0, 499, 342, 0, 1760698862,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760698862), 0.0, 0.0, 14, '', 0, 0, '1760698831_991460_1389741301_ac3a1eaca2c726ebf3d3f70772f36fd2_rle_crop_4000359729_0.png', 0, 56, 97, 0, 1760698862,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760698862), 0.0, 0.0, 14, '', 0, 0, '1760698831_991460_1389741295_554ecea9e330da1bfca42e582f4e5c62_rle_crop_4000359733_0.png', 0, 970, 957, 0, 1760698862,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760698862), 0.0, 0.0, 14, '', 0, 0, '1760698831_991460_1389741295_554ecea9e330da1bfca42e582f4e5c62_rle_crop_4000359731_0.png', 0, 117, 70, 0, 1760698862,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760698862), 0.0, 0.0, 14, '', 0, 0, '1760698831_991460_1389740938_52b53bba96c4d06f0b3f5eed96049280_rle_crop_4000359737_0.png', 0, 74, 72, 0, 1760698862,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760698862), 0.0, 0.0, 14, '', 0, 0, '1760698831_991460_1389740931_a146f987bca174f4356c3dbf6b6ed245_rle_crop_4000359743_0.png', 0, 112, 132, 0, 1760698862,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 7 photos in the portfolio 3736932 time of upload the photos Elapsed time : 3.143209934234619 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 9 About to insert : list_path_to_insert length 9 new photo from crops ! About to upload 9 photos upload in portfolio : 3736932 init cache_photo without model_param we have 9 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1760698872_991460 INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760698873), 0.0, 0.0, 14, '', 0, 0, '1760698831_991460_1389741301_ac3a1eaca2c726ebf3d3f70772f36fd2_rle_crop_4000359728_0.png', 0, 1024, 986, 0, 1760698873,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760698873), 0.0, 0.0, 14, '', 0, 0, '1760698831_991460_1389741301_ac3a1eaca2c726ebf3d3f70772f36fd2_rle_crop_4000359727_0.png', 0, 80, 154, 0, 1760698873,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760698873), 0.0, 0.0, 14, '', 0, 0, '1760698831_991460_1389741295_554ecea9e330da1bfca42e582f4e5c62_rle_crop_4000359732_0.png', 0, 309, 498, 0, 1760698873,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760698873), 0.0, 0.0, 14, '', 0, 0, '1760698831_991460_1389741290_ad821ed2fb15b8fbdf2216f83c832aba_rle_crop_4000359734_0.png', 0, 200, 129, 0, 1760698873,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760698873), 0.0, 0.0, 14, '', 0, 0, '1760698831_991460_1389740938_52b53bba96c4d06f0b3f5eed96049280_rle_crop_4000359736_0.png', 0, 102, 115, 0, 1760698873,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760698873), 0.0, 0.0, 14, '', 0, 0, '1760698831_991460_1389740938_52b53bba96c4d06f0b3f5eed96049280_rle_crop_4000359740_0.png', 0, 224, 242, 0, 1760698873,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760698873), 0.0, 0.0, 14, '', 0, 0, '1760698831_991460_1389740938_52b53bba96c4d06f0b3f5eed96049280_rle_crop_4000359738_0.png', 0, 147, 224, 0, 1760698873,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760698873), 0.0, 0.0, 14, '', 0, 0, '1760698831_991460_1389740931_a146f987bca174f4356c3dbf6b6ed245_rle_crop_4000359744_0.png', 0, 1118, 978, 0, 1760698873,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1760698873), 0.0, 0.0, 14, '', 0, 0, '1760698831_991460_1389740931_a146f987bca174f4356c3dbf6b6ed245_rle_crop_4000359741_0.png', 0, 165, 133, 0, 1760698873,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 9 photos in the portfolio 3736932 time of upload the photos Elapsed time : 3.208531379699707 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1389741311, 1389741310, 1389741307, 1389741301, 1389741295, 1389741290, 1389740938, 1389740931] Looping around the photos to save general results len do output : 22 /1389796199Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389796200Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389796201Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389796202Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389796203Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389796204Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389796316Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389796330Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389796344Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389796358Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389796371Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389796385Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389796391Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389796399Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389796412Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389796427Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389796433Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389796434Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389796447Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389796458Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389796470Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1389796484Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741311', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741310', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741307', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741301', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741295', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741290', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389740938', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389740931', None, None, None, None, None, '3954038') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 74 time used for this insertion : 0.037859201431274414 save_final save missing photos in datou_result : time spend for datou_step_exec : 21.38385844230652 time spend to save output : 0.03890252113342285 total time spend for step 2 : 21.42276096343994 step3:rle_unique_nms_with_priority Fri Oct 17 13:01:14 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 22 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.4617018699645996 time for calcul the mask position with numpy : 0.1061546802520752 nb_pixel_total : 1724644 time to create 1 rle with new method : 0.09164667129516602 time for calcul the mask position with numpy : 0.010123014450073242 nb_pixel_total : 346277 time to create 1 rle with new method : 0.17071032524108887 time for calcul the mask position with numpy : 0.006562709808349609 nb_pixel_total : 2679 time to create 1 rle with old method : 0.003120899200439453 create new chi : 0.40459489822387695 time to delete rle : 0.12117528915405273 batch 1 Loaded 5 chid ids of type : 3594 +++++++Number RLEs to save : 4188 TO DO : save crop sub photo not yet done ! save time : 0.5936791896820068 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.2674083709716797 time for calcul the mask position with numpy : 0.047718048095703125 nb_pixel_total : 2063688 time to create 1 rle with new method : 0.10991191864013672 time for calcul the mask position with numpy : 0.007462978363037109 nb_pixel_total : 9912 time to create 1 rle with old method : 0.012750387191772461 create new chi : 0.18507075309753418 time to delete rle : 0.0003170967102050781 batch 1 Loaded 3 chid ids of type : 3594 ++++++Number RLEs to save : 1688 TO DO : save crop sub photo not yet done ! save time : 0.3370499610900879 No data in photo_id : 1389741307 nb_obj : 5 nb_hashtags : 3 time to prepare the origin masks : 0.23738574981689453 time for calcul the mask position with numpy : 0.04817461967468262 nb_pixel_total : 1305660 time to create 1 rle with new method : 0.2315974235534668 time for calcul the mask position with numpy : 0.0071773529052734375 nb_pixel_total : 8980 time to create 1 rle with old method : 0.01022481918334961 time for calcul the mask position with numpy : 0.006291627883911133 nb_pixel_total : 3675 time to create 1 rle with old method : 0.0043904781341552734 time for calcul the mask position with numpy : 0.012247562408447266 nb_pixel_total : 739451 time to create 1 rle with new method : 0.3144950866699219 time for calcul the mask position with numpy : 0.007354736328125 nb_pixel_total : 10655 time to create 1 rle with old method : 0.012358427047729492 time for calcul the mask position with numpy : 0.007624626159667969 nb_pixel_total : 5179 time to create 1 rle with old method : 0.0060367584228515625 create new chi : 0.6857326030731201 time to delete rle : 0.0011181831359863281 batch 1 Loaded 11 chid ids of type : 3594 +++++Number RLEs to save : 4032 TO DO : save crop sub photo not yet done ! save time : 0.5959136486053467 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.0726461410522461 time for calcul the mask position with numpy : 0.028220176696777344 nb_pixel_total : 1302838 time to create 1 rle with new method : 0.11704277992248535 time for calcul the mask position with numpy : 0.017293930053710938 nb_pixel_total : 664335 time to create 1 rle with new method : 0.04159736633300781 time for calcul the mask position with numpy : 0.008470773696899414 nb_pixel_total : 99883 time to create 1 rle with old method : 0.11173439025878906 time for calcul the mask position with numpy : 0.007853031158447266 nb_pixel_total : 6544 time to create 1 rle with old method : 0.0076792240142822266 create new chi : 0.3412027359008789 time to delete rle : 0.0008847713470458984 batch 1 Loaded 7 chid ids of type : 3594 +++Number RLEs to save : 4148 TO DO : save crop sub photo not yet done ! save time : 0.6205987930297852 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.06199836730957031 time for calcul the mask position with numpy : 0.034810781478881836 nb_pixel_total : 2050279 time to create 1 rle with new method : 0.04280996322631836 time for calcul the mask position with numpy : 0.007980585098266602 nb_pixel_total : 8231 time to create 1 rle with old method : 0.009336709976196289 time for calcul the mask position with numpy : 0.008018016815185547 nb_pixel_total : 15090 time to create 1 rle with old method : 0.017220497131347656 create new chi : 0.12069511413574219 time to delete rle : 0.0004520416259765625 batch 1 Loaded 5 chid ids of type : 3594 ++++Number RLEs to save : 1592 TO DO : save crop sub photo not yet done ! save time : 0.327345609664917 nb_obj : 5 nb_hashtags : 3 time to prepare the origin masks : 0.29132819175720215 time for calcul the mask position with numpy : 0.10952544212341309 nb_pixel_total : 1999175 time to create 1 rle with new method : 0.12213969230651855 time for calcul the mask position with numpy : 0.009373664855957031 nb_pixel_total : 28931 time to create 1 rle with old method : 0.03286433219909668 time for calcul the mask position with numpy : 0.008078813552856445 nb_pixel_total : 7465 time to create 1 rle with old method : 0.00858759880065918 time for calcul the mask position with numpy : 0.008133172988891602 nb_pixel_total : 24616 time to create 1 rle with old method : 0.028024911880493164 time for calcul the mask position with numpy : 0.008051395416259766 nb_pixel_total : 3895 time to create 1 rle with old method : 0.004605293273925781 time for calcul the mask position with numpy : 0.008153438568115234 nb_pixel_total : 9518 time to create 1 rle with old method : 0.010748147964477539 create new chi : 0.3683629035949707 time to delete rle : 0.0007503032684326172 batch 1 Loaded 11 chid ids of type : 3594 +++++Number RLEs to save : 2564 TO DO : save crop sub photo not yet done ! save time : 0.4628159999847412 nb_obj : 4 nb_hashtags : 3 time to prepare the origin masks : 0.10749340057373047 time for calcul the mask position with numpy : 0.04835009574890137 nb_pixel_total : 1310746 time to create 1 rle with new method : 0.12209367752075195 time for calcul the mask position with numpy : 0.017858505249023438 nb_pixel_total : 736276 time to create 1 rle with new method : 0.08198213577270508 time for calcul the mask position with numpy : 0.006937980651855469 nb_pixel_total : 8934 time to create 1 rle with old method : 0.01014566421508789 time for calcul the mask position with numpy : 0.006917476654052734 nb_pixel_total : 5826 time to create 1 rle with old method : 0.008657455444335938 time for calcul the mask position with numpy : 0.007025003433227539 nb_pixel_total : 11818 time to create 1 rle with old method : 0.013633489608764648 create new chi : 0.33139777183532715 time to delete rle : 0.000820159912109375 batch 1 Loaded 9 chid ids of type : 3594 ++++++++++++Number RLEs to save : 4829 TO DO : save crop sub photo not yet done ! save time : 0.6598856449127197 map_output_result : {1389741311: (0.0, 'Should be the crop_list due to order', 0), 1389741310: (0.0, 'Should be the crop_list due to order', 0), 1389741307: (0.0, 'Should be the crop_list due to order', 0.0), 1389741301: (0.0, 'Should be the crop_list due to order', 0), 1389741295: (0.0, 'Should be the crop_list due to order', 0), 1389741290: (0.0, 'Should be the crop_list due to order', 0), 1389740938: (0.0, 'Should be the crop_list due to order', 0), 1389740931: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1389741311, 1389741310, 1389741307, 1389741301, 1389741295, 1389741290, 1389740938, 1389740931] Looping around the photos to save general results len do output : 8 /1389741311.Didn't retrieve data . /1389741310.Didn't retrieve data . /1389741307.Didn't retrieve data . /1389741301.Didn't retrieve data . /1389741295.Didn't retrieve data . /1389741290.Didn't retrieve data . /1389740938.Didn't retrieve data . /1389740931.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741311', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741310', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741307', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741301', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741295', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741290', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389740938', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389740931', None, None, None, None, None, '3954038') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 24 time used for this insertion : 0.03806281089782715 save_final save missing photos in datou_result : time spend for datou_step_exec : 8.088907718658447 time spend to save output : 0.038452863693237305 total time spend for step 3 : 8.127360582351685 step4:ventilate_hashtags_in_portfolio Fri Oct 17 13:01:23 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 27877917 get user id for portfolio 27877917 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27877917 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('carton','flou','pehd','background','autre','environnement','mal_croppe','papier','pet_fonce','pet_clair','metal')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27877917 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('carton','flou','pehd','background','autre','environnement','mal_croppe','papier','pet_fonce','pet_clair','metal')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27877917 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('carton','flou','pehd','background','autre','environnement','mal_croppe','papier','pet_fonce','pet_clair','metal')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://marlene.fotonower.com/velours/27879962,27879963,27879964,27879965,27879966,27879967,27879968,27879969,27879970,27879971,27879972?tags=carton,flou,pehd,background,autre,environnement,mal_croppe,papier,pet_fonce,pet_clair,metal Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1389741311, 1389741310, 1389741307, 1389741301, 1389741295, 1389741290, 1389740938, 1389740931] Looping around the photos to save general results len do output : 1 /27877917. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741311', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741310', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741307', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741301', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741295', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741290', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389740938', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389740931', None, None, None, None, None, '3954038') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.03636479377746582 save_final save missing photos in datou_result : time spend for datou_step_exec : 7.084099054336548 time spend to save output : 0.036684513092041016 total time spend for step 4 : 7.120783567428589 step5:final Fri Oct 17 13:01:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1389741311: ('0.16626699942129627',), 1389741310: ('0.16626699942129627',), 1389741307: ('0.16626699942129627',), 1389741301: ('0.16626699942129627',), 1389741295: ('0.16626699942129627',), 1389741290: ('0.16626699942129627',), 1389740938: ('0.16626699942129627',), 1389740931: ('0.16626699942129627',)} new output for save of step final : {1389741311: ('0.16626699942129627',), 1389741310: ('0.16626699942129627',), 1389741307: ('0.16626699942129627',), 1389741301: ('0.16626699942129627',), 1389741295: ('0.16626699942129627',), 1389741290: ('0.16626699942129627',), 1389740938: ('0.16626699942129627',), 1389740931: ('0.16626699942129627',)} [1389741311, 1389741310, 1389741307, 1389741301, 1389741295, 1389741290, 1389740938, 1389740931] Looping around the photos to save general results len do output : 8 /1389741311.Didn't retrieve data . /1389741310.Didn't retrieve data . /1389741307.Didn't retrieve data . /1389741301.Didn't retrieve data . /1389741295.Didn't retrieve data . /1389741290.Didn't retrieve data . /1389740938.Didn't retrieve data . /1389740931.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741311', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741310', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741307', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741301', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741295', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741290', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389740938', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389740931', None, None, None, None, None, '3954038') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 24 time used for this insertion : 0.0389704704284668 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.4032163619995117 time spend to save output : 0.039467811584472656 total time spend for step 5 : 0.4426841735839844 step6:blur_detection Fri Oct 17 13:01:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1760698831_991460_1389741311_b4f8306dc65c3bdb7ee7c5acc8966e42.jpg resize: (1080, 1920) 1389741311 -1.1042838974220448 treat image : temp/1760698831_991460_1389741310_b6565a6d046262f99bd2fe5136643b79.jpg resize: (1080, 1920) 1389741310 1.7880494093812962 treat image : temp/1760698831_991460_1389741307_f881868bcc2a2368701d9596ff377de7.jpg resize: (1080, 1920) 1389741307 -0.9657728119989031 treat image : temp/1760698831_991460_1389741301_ac3a1eaca2c726ebf3d3f70772f36fd2.jpg resize: (1080, 1920) 1389741301 0.4842239957613866 treat image : temp/1760698831_991460_1389741295_554ecea9e330da1bfca42e582f4e5c62.jpg resize: (1080, 1920) 1389741295 -0.8392259073050735 treat image : temp/1760698831_991460_1389741290_ad821ed2fb15b8fbdf2216f83c832aba.jpg resize: (1080, 1920) 1389741290 0.344622323635863 treat image : temp/1760698831_991460_1389740938_52b53bba96c4d06f0b3f5eed96049280.jpg resize: (1080, 1920) 1389740938 -1.6290015317579123 treat image : temp/1760698831_991460_1389740931_a146f987bca174f4356c3dbf6b6ed245.jpg resize: (1080, 1920) 1389740931 -0.6460233026392749 treat image : temp/1760698831_991460_1389741311_b4f8306dc65c3bdb7ee7c5acc8966e42_rle_crop_4000359723_0.png resize: (40, 93) 1389796199 -2.0541550204466095 treat image : temp/1760698831_991460_1389741301_ac3a1eaca2c726ebf3d3f70772f36fd2_rle_crop_4000359730_0.png resize: (121, 135) 1389796200 -2.48513754567463 treat image : temp/1760698831_991460_1389741301_ac3a1eaca2c726ebf3d3f70772f36fd2_rle_crop_4000359726_0.png resize: (99, 77) 1389796201 -2.09670731428534 treat image : temp/1760698831_991460_1389741290_ad821ed2fb15b8fbdf2216f83c832aba_rle_crop_4000359735_0.png resize: (112, 106) 1389796202 -0.07109747995156052 treat image : temp/1760698831_991460_1389740938_52b53bba96c4d06f0b3f5eed96049280_rle_crop_4000359739_0.png resize: (88, 115) 1389796203 -1.54422435213745 treat image : temp/1760698831_991460_1389740931_a146f987bca174f4356c3dbf6b6ed245_rle_crop_4000359742_0.png resize: (123, 67) 1389796204 -0.3013589794580313 treat image : temp/1760698831_991460_1389741311_b4f8306dc65c3bdb7ee7c5acc8966e42_rle_crop_4000359724_0.png resize: (887, 876) 1389796316 -0.9087890683154602 treat image : temp/1760698831_991460_1389741310_b6565a6d046262f99bd2fe5136643b79_rle_crop_4000359725_0.png resize: (342, 499) 1389796330 0.4309223058069269 treat image : temp/1760698831_991460_1389741301_ac3a1eaca2c726ebf3d3f70772f36fd2_rle_crop_4000359729_0.png resize: (97, 56) 1389796344 -1.4139033254509465 treat image : temp/1760698831_991460_1389741295_554ecea9e330da1bfca42e582f4e5c62_rle_crop_4000359733_0.png resize: (957, 970) 1389796358 -0.31075486013044634 treat image : temp/1760698831_991460_1389741295_554ecea9e330da1bfca42e582f4e5c62_rle_crop_4000359731_0.png resize: (70, 117) 1389796371 0.35501449060242224 treat image : temp/1760698831_991460_1389740938_52b53bba96c4d06f0b3f5eed96049280_rle_crop_4000359737_0.png resize: (72, 74) 1389796385 0.9020754923356104 treat image : temp/1760698831_991460_1389740931_a146f987bca174f4356c3dbf6b6ed245_rle_crop_4000359743_0.png resize: (132, 112) 1389796391 -2.0511305544164884 treat image : temp/1760698831_991460_1389741301_ac3a1eaca2c726ebf3d3f70772f36fd2_rle_crop_4000359728_0.png resize: (986, 1024) 1389796399 -0.18939698525643717 treat image : temp/1760698831_991460_1389741301_ac3a1eaca2c726ebf3d3f70772f36fd2_rle_crop_4000359727_0.png resize: (154, 80) 1389796412 2.8115360378279175 treat image : temp/1760698831_991460_1389741295_554ecea9e330da1bfca42e582f4e5c62_rle_crop_4000359732_0.png resize: (498, 309) 1389796427 0.39185477366403987 treat image : temp/1760698831_991460_1389741290_ad821ed2fb15b8fbdf2216f83c832aba_rle_crop_4000359734_0.png resize: (129, 200) 1389796433 -0.44337902416576 treat image : temp/1760698831_991460_1389740938_52b53bba96c4d06f0b3f5eed96049280_rle_crop_4000359736_0.png resize: (115, 102) 1389796434 -4.090292028221824 treat image : temp/1760698831_991460_1389740938_52b53bba96c4d06f0b3f5eed96049280_rle_crop_4000359740_0.png resize: (242, 224) 1389796447 -1.4255101778933434 treat image : temp/1760698831_991460_1389740938_52b53bba96c4d06f0b3f5eed96049280_rle_crop_4000359738_0.png resize: (224, 147) 1389796458 -1.4057439746316855 treat image : temp/1760698831_991460_1389740931_a146f987bca174f4356c3dbf6b6ed245_rle_crop_4000359744_0.png resize: (978, 1118) 1389796470 0.015994953753911648 treat image : temp/1760698831_991460_1389740931_a146f987bca174f4356c3dbf6b6ed245_rle_crop_4000359741_0.png resize: (133, 165) 1389796484 -2.464193722185127 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 30 time used for this insertion : 0.03643298149108887 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 30 time used for this insertion : 0.03755474090576172 save missing photos in datou_result : time spend for datou_step_exec : 8.889734983444214 time spend to save output : 0.0919337272644043 total time spend for step 6 : 8.981668710708618 step7:brightness Fri Oct 17 13:01:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1760698831_991460_1389741311_b4f8306dc65c3bdb7ee7c5acc8966e42.jpg treat image : temp/1760698831_991460_1389741310_b6565a6d046262f99bd2fe5136643b79.jpg treat image : temp/1760698831_991460_1389741307_f881868bcc2a2368701d9596ff377de7.jpg treat image : temp/1760698831_991460_1389741301_ac3a1eaca2c726ebf3d3f70772f36fd2.jpg treat image : temp/1760698831_991460_1389741295_554ecea9e330da1bfca42e582f4e5c62.jpg treat image : temp/1760698831_991460_1389741290_ad821ed2fb15b8fbdf2216f83c832aba.jpg treat image : temp/1760698831_991460_1389740938_52b53bba96c4d06f0b3f5eed96049280.jpg treat image : temp/1760698831_991460_1389740931_a146f987bca174f4356c3dbf6b6ed245.jpg treat image : temp/1760698831_991460_1389741311_b4f8306dc65c3bdb7ee7c5acc8966e42_rle_crop_4000359723_0.png treat image : temp/1760698831_991460_1389741301_ac3a1eaca2c726ebf3d3f70772f36fd2_rle_crop_4000359730_0.png treat image : temp/1760698831_991460_1389741301_ac3a1eaca2c726ebf3d3f70772f36fd2_rle_crop_4000359726_0.png treat image : temp/1760698831_991460_1389741290_ad821ed2fb15b8fbdf2216f83c832aba_rle_crop_4000359735_0.png treat image : temp/1760698831_991460_1389740938_52b53bba96c4d06f0b3f5eed96049280_rle_crop_4000359739_0.png treat image : temp/1760698831_991460_1389740931_a146f987bca174f4356c3dbf6b6ed245_rle_crop_4000359742_0.png treat image : temp/1760698831_991460_1389741311_b4f8306dc65c3bdb7ee7c5acc8966e42_rle_crop_4000359724_0.png treat image : temp/1760698831_991460_1389741310_b6565a6d046262f99bd2fe5136643b79_rle_crop_4000359725_0.png treat image : temp/1760698831_991460_1389741301_ac3a1eaca2c726ebf3d3f70772f36fd2_rle_crop_4000359729_0.png treat image : temp/1760698831_991460_1389741295_554ecea9e330da1bfca42e582f4e5c62_rle_crop_4000359733_0.png treat image : temp/1760698831_991460_1389741295_554ecea9e330da1bfca42e582f4e5c62_rle_crop_4000359731_0.png treat image : temp/1760698831_991460_1389740938_52b53bba96c4d06f0b3f5eed96049280_rle_crop_4000359737_0.png treat image : temp/1760698831_991460_1389740931_a146f987bca174f4356c3dbf6b6ed245_rle_crop_4000359743_0.png treat image : temp/1760698831_991460_1389741301_ac3a1eaca2c726ebf3d3f70772f36fd2_rle_crop_4000359728_0.png treat image : temp/1760698831_991460_1389741301_ac3a1eaca2c726ebf3d3f70772f36fd2_rle_crop_4000359727_0.png treat image : temp/1760698831_991460_1389741295_554ecea9e330da1bfca42e582f4e5c62_rle_crop_4000359732_0.png treat image : temp/1760698831_991460_1389741290_ad821ed2fb15b8fbdf2216f83c832aba_rle_crop_4000359734_0.png treat image : temp/1760698831_991460_1389740938_52b53bba96c4d06f0b3f5eed96049280_rle_crop_4000359736_0.png treat image : temp/1760698831_991460_1389740938_52b53bba96c4d06f0b3f5eed96049280_rle_crop_4000359740_0.png treat image : temp/1760698831_991460_1389740938_52b53bba96c4d06f0b3f5eed96049280_rle_crop_4000359738_0.png treat image : temp/1760698831_991460_1389740931_a146f987bca174f4356c3dbf6b6ed245_rle_crop_4000359744_0.png treat image : temp/1760698831_991460_1389740931_a146f987bca174f4356c3dbf6b6ed245_rle_crop_4000359741_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 30 time used for this insertion : 0.03947114944458008 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 30 time used for this insertion : 0.03693199157714844 save missing photos in datou_result : time spend for datou_step_exec : 2.600787401199341 time spend to save output : 0.09467506408691406 total time spend for step 7 : 2.695462465286255 step8:velours_tree Fri Oct 17 13:01:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.24474620819091797 time spend to save output : 6.532669067382812e-05 total time spend for step 8 : 0.2448115348815918 step9:send_mail_cod Fri Oct 17 13:01:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P27877917_17-10-2025_13_01_42.pdf 27879962 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette278799621760698902 27879963 imagette278799631760698903 27879964 imagette278799641760698903 27879965 imagette278799651760698903 27879966 imagette278799661760698903 27879968 imagette278799681760698903 27879969 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette278799691760698903 27879970 imagette278799701760698903 27879971 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette278799711760698903 27879972 imagette278799721760698904 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=27877917 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://marlene.fotonower.com/velours/27879962,27879963,27879964,27879965,27879966,27879967,27879968,27879969,27879970,27879971,27879972?tags=carton,flou,pehd,background,autre,environnement,mal_croppe,papier,pet_fonce,pet_clair,metal args[1389741311] : ((1389741311, -1.1042838974220448, 492688767), (1389741311, 0.5161376781896568, 2107752395), '0.16626699942129627') We are sending mail with results at report@fotonower.com args[1389741310] : ((1389741310, 1.7880494093812962, 492688767), (1389741310, 0.5472572402316634, 2107752395), '0.16626699942129627') We are sending mail with results at report@fotonower.com args[1389741307] : ((1389741307, -0.9657728119989031, 492688767), (1389741307, 0.7219986642134433, 2107752395), '0.16626699942129627') We are sending mail with results at report@fotonower.com args[1389741301] : ((1389741301, 0.4842239957613866, 492688767), (1389741301, 0.4022455754149829, 2107752395), '0.16626699942129627') We are sending mail with results at report@fotonower.com args[1389741295] : ((1389741295, -0.8392259073050735, 492688767), (1389741295, 0.32520598882249674, 2107752395), '0.16626699942129627') We are sending mail with results at report@fotonower.com args[1389741290] : ((1389741290, 0.344622323635863, 492688767), (1389741290, 0.9262045874760587, 2107752395), '0.16626699942129627') We are sending mail with results at report@fotonower.com args[1389740938] : ((1389740938, -1.6290015317579123, 492688767), (1389740938, 0.5844358821930052, 2107752395), '0.16626699942129627') We are sending mail with results at report@fotonower.com args[1389740931] : ((1389740931, -0.6460233026392749, 492688767), (1389740931, 0.6799674052949491, 2107752395), '0.16626699942129627') We are sending mail with results at report@fotonower.com refus_total : 0.16626699942129627 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos_view ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=27877917 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27877917_17-10-2025_13_01_42.pdf results_Auto_P27877917_17-10-2025_13_01_42.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27877917_17-10-2025_13_01_42.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','27877917','results_Auto_P27877917_17-10-2025_13_01_42.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27877917_17-10-2025_13_01_42.pdf','pdf','','0.34','0.16626699942129627') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/27877917

https://www.fotonower.com/image?json=false&list_photos_id=1389741311
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1389741310
La photo est trop floue, merci de reprendre une photo.(avec le score = 1.7880494093812962)
https://www.fotonower.com/image?json=false&list_photos_id=1389741307
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1389741301
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1389741295
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1389741290
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1389740938
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1389740931
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 16.63%
Veuillez trouver les photos des contaminants.

exemples de contaminants: carton: https://www.fotonower.com/view/27879962?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/27879969?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/27879971?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27877917_17-10-2025_13_01_42.pdf.

Lien vers velours :https://marlene.fotonower.com/velours/27879962,27879963,27879964,27879965,27879966,27879967,27879968,27879969,27879970,27879971,27879972?tags=carton,flou,pehd,background,autre,environnement,mal_croppe,papier,pet_fonce,pet_clair,metal.


L'équipe Fotonower 202 b'' Server: nginx Date: Fri, 17 Oct 2025 11:01:46 GMT Content-Length: 0 Connection: close X-Message-Id: ni5kz1cmRaaTgaPX-bIe4g Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1389741311, 1389741310, 1389741307, 1389741301, 1389741295, 1389741290, 1389740938, 1389740931] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741311', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741310', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741307', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741301', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741295', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741290', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389740938', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389740931', None, None, None, None, None, '3954038') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 8 time used for this insertion : 0.03743600845336914 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.335582733154297 time spend to save output : 0.03765511512756348 total time spend for step 9 : 4.37323784828186 step10:split_time_score Fri Oct 17 13:01:46 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('10', 8),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 16102025 27877917 Nombre de photos uploadées : 8 / 23040 (0%) 16102025 27877917 Nombre de photos taguées (types de déchets): 0 / 8 (0%) 16102025 27877917 Nombre de photos taguées (volume) : 0 / 8 (0%) elapsed_time : load_data_split_time_score 2.1457672119140625e-06 elapsed_time : order_list_meta_photo_and_scores 6.4373016357421875e-06 ???????? elapsed_time : fill_and_build_computed_from_old_data 0.0004718303680419922 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.636970043182373 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.16626699942129627 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27877917_17-10-2025_13_01_42.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27877917 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27877917 AND mptpi.`type`=3594 To do Qualite : 0.02519941165123456 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27877920_17-10-2025_12_52_23.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27877920 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27877920 AND mptpi.`type`=3594 To do Qualite : 0.10820650077160499 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27868451_16-10-2025_13_42_12.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27868451 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27868451 AND mptpi.`type`=3594 To do Qualite : 0.061227575231481486 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27868453_16-10-2025_12_41_58.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27868453 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27868453 AND mptpi.`type`=3594 To do Qualite : 0.020004677854938268 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27877923_17-10-2025_12_41_31.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27877923 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27877923 AND mptpi.`type`=3594 To do Qualite : 0.05956004050925926 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27877954_17-10-2025_12_31_26.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27877954 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27877954 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'16102025': {'nb_upload': 8, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1389741311, 1389741310, 1389741307, 1389741301, 1389741295, 1389741290, 1389740938, 1389740931] Looping around the photos to save general results len do output : 1 /27877917Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741311', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741310', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741307', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741301', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741295', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389741290', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389740938', None, None, None, None, None, '3954038') ('3318', None, None, None, None, None, None, None, '3954038') ('3318', '27877917', '1389740931', None, None, None, None, None, '3954038') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.07755470275878906 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.4765524864196777 time spend to save output : 0.07783842086791992 total time spend for step 10 : 2.5543909072875977 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 8 set_done_treatment 40.64user 12.72system 1:23.40elapsed 63%CPU (0avgtext+0avgdata 2699184maxresident)k 15616inputs+22504outputs (228major+1221600minor)pagefaults 0swaps