python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 739567 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['3568032'] with mtr_portfolio_ids : ['26116300'] and first list_photo_ids : [] new path : /proc/739567/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 13 ; length of list_pids : 13 ; length of list_args : 13 time to download the photos : 1.617614507675171 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Thu Aug 21 10:40:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10615 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-08-21 10:40:33.580249: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-08-21 10:40:33.587520: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-08-21 10:40:33.589064: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fe690000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-08-21 10:40:33.589104: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-08-21 10:40:33.591322: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-08-21 10:40:33.702868: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x2e0a0920 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-08-21 10:40:33.702925: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-08-21 10:40:33.704713: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-21 10:40:33.705198: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-21 10:40:33.707110: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-21 10:40:33.708703: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-21 10:40:33.708999: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-21 10:40:33.710871: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-21 10:40:33.711841: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-21 10:40:33.716123: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-21 10:40:33.717868: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-21 10:40:33.717939: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-21 10:40:33.718856: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-08-21 10:40:33.718875: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-08-21 10:40:33.718885: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-08-21 10:40:33.720563: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9689 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-08-21 10:40:33.966084: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-21 10:40:33.966162: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-21 10:40:33.966183: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-21 10:40:33.966202: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-21 10:40:33.966220: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-21 10:40:33.966238: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-21 10:40:33.966255: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-21 10:40:33.966273: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-21 10:40:33.967829: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-21 10:40:33.969178: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-21 10:40:33.969215: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-21 10:40:33.969233: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-21 10:40:33.969250: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-21 10:40:33.969269: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-21 10:40:33.969289: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-21 10:40:33.969305: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-21 10:40:33.969322: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-21 10:40:33.970860: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-21 10:40:33.970890: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-08-21 10:40:33.970900: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-08-21 10:40:33.970909: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-08-21 10:40:33.972545: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9689 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-08-21 10:40:41.674086: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-21 10:40:41.938449: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 13 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 39.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 31.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 49.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 31.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 27.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 36.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 21 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 33.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 48.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 49.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 54.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 54.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 38.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 43.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 Detection mask done ! Trying to reset tf kernel 740238 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 747 tf kernel not reseted sub process len(results) : 13 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 13 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1546 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0003027915954589844 nb_pixel_total : 11855 time to create 1 rle with old method : 0.013451576232910156 length of segment : 154 time for calcul the mask position with numpy : 0.0001926422119140625 nb_pixel_total : 8658 time to create 1 rle with old method : 0.009622335433959961 length of segment : 173 time for calcul the mask position with numpy : 0.00027561187744140625 nb_pixel_total : 10382 time to create 1 rle with old method : 0.011847257614135742 length of segment : 148 time for calcul the mask position with numpy : 0.00021147727966308594 nb_pixel_total : 12464 time to create 1 rle with old method : 0.014296531677246094 length of segment : 120 time for calcul the mask position with numpy : 0.000324249267578125 nb_pixel_total : 24832 time to create 1 rle with old method : 0.027408838272094727 length of segment : 201 time for calcul the mask position with numpy : 0.00013828277587890625 nb_pixel_total : 7353 time to create 1 rle with old method : 0.008306026458740234 length of segment : 80 time for calcul the mask position with numpy : 0.00022459030151367188 nb_pixel_total : 9989 time to create 1 rle with old method : 0.011336326599121094 length of segment : 173 time for calcul the mask position with numpy : 0.0012791156768798828 nb_pixel_total : 76044 time to create 1 rle with old method : 0.08398127555847168 length of segment : 434 time for calcul the mask position with numpy : 8.487701416015625e-05 nb_pixel_total : 3528 time to create 1 rle with old method : 0.004174709320068359 length of segment : 42 time for calcul the mask position with numpy : 0.0003070831298828125 nb_pixel_total : 19794 time to create 1 rle with old method : 0.022495269775390625 length of segment : 167 time for calcul the mask position with numpy : 8.58306884765625e-05 nb_pixel_total : 4066 time to create 1 rle with old method : 0.004820585250854492 length of segment : 61 time for calcul the mask position with numpy : 0.0002110004425048828 nb_pixel_total : 15551 time to create 1 rle with old method : 0.017482280731201172 length of segment : 128 time for calcul the mask position with numpy : 0.00013566017150878906 nb_pixel_total : 8177 time to create 1 rle with old method : 0.00946950912475586 length of segment : 142 time for calcul the mask position with numpy : 0.00022339820861816406 nb_pixel_total : 17128 time to create 1 rle with old method : 0.01904463768005371 length of segment : 164 time for calcul the mask position with numpy : 8.845329284667969e-05 nb_pixel_total : 3710 time to create 1 rle with old method : 0.0043103694915771484 length of segment : 85 time for calcul the mask position with numpy : 0.0018723011016845703 nb_pixel_total : 123802 time to create 1 rle with old method : 0.13432908058166504 length of segment : 579 time for calcul the mask position with numpy : 0.023120641708374023 nb_pixel_total : 726622 time to create 1 rle with new method : 0.03835320472717285 length of segment : 974 time for calcul the mask position with numpy : 0.002207040786743164 nb_pixel_total : 96044 time to create 1 rle with old method : 0.10784435272216797 length of segment : 478 time for calcul the mask position with numpy : 9.584426879882812e-05 nb_pixel_total : 2498 time to create 1 rle with old method : 0.003138303756713867 length of segment : 78 time for calcul the mask position with numpy : 0.0001857280731201172 nb_pixel_total : 11080 time to create 1 rle with old method : 0.013041257858276367 length of segment : 123 time for calcul the mask position with numpy : 5.555152893066406e-05 nb_pixel_total : 1436 time to create 1 rle with old method : 0.0017724037170410156 length of segment : 50 time for calcul the mask position with numpy : 9.512901306152344e-05 nb_pixel_total : 3348 time to create 1 rle with old method : 0.004105091094970703 length of segment : 67 time for calcul the mask position with numpy : 7.05718994140625e-05 nb_pixel_total : 1548 time to create 1 rle with old method : 0.0019528865814208984 length of segment : 60 time for calcul the mask position with numpy : 0.00017070770263671875 nb_pixel_total : 9094 time to create 1 rle with old method : 0.01072835922241211 length of segment : 103 time for calcul the mask position with numpy : 0.0002338886260986328 nb_pixel_total : 14468 time to create 1 rle with old method : 0.016872406005859375 length of segment : 118 time for calcul the mask position with numpy : 0.00024175643920898438 nb_pixel_total : 13769 time to create 1 rle with old method : 0.01587820053100586 length of segment : 191 time for calcul the mask position with numpy : 0.00016546249389648438 nb_pixel_total : 5902 time to create 1 rle with old method : 0.006888628005981445 length of segment : 121 time for calcul the mask position with numpy : 0.0003256797790527344 nb_pixel_total : 13647 time to create 1 rle with old method : 0.016083240509033203 length of segment : 106 time for calcul the mask position with numpy : 0.0003483295440673828 nb_pixel_total : 8919 time to create 1 rle with old method : 0.010350227355957031 length of segment : 122 time for calcul the mask position with numpy : 0.00023126602172851562 nb_pixel_total : 4719 time to create 1 rle with old method : 0.005713462829589844 length of segment : 101 time for calcul the mask position with numpy : 0.013246774673461914 nb_pixel_total : 642568 time to create 1 rle with new method : 0.7079360485076904 length of segment : 1027 time for calcul the mask position with numpy : 0.0013628005981445312 nb_pixel_total : 71586 time to create 1 rle with old method : 0.0768587589263916 length of segment : 468 time for calcul the mask position with numpy : 0.0007381439208984375 nb_pixel_total : 28672 time to create 1 rle with old method : 0.03228616714477539 length of segment : 205 time for calcul the mask position with numpy : 0.0001633167266845703 nb_pixel_total : 7152 time to create 1 rle with old method : 0.008262157440185547 length of segment : 91 time for calcul the mask position with numpy : 0.0020813941955566406 nb_pixel_total : 78590 time to create 1 rle with old method : 0.0866847038269043 length of segment : 499 time for calcul the mask position with numpy : 0.00024700164794921875 nb_pixel_total : 4933 time to create 1 rle with old method : 0.005594015121459961 length of segment : 111 time for calcul the mask position with numpy : 0.0006000995635986328 nb_pixel_total : 25515 time to create 1 rle with old method : 0.029077529907226562 length of segment : 163 time for calcul the mask position with numpy : 0.0042955875396728516 nb_pixel_total : 189393 time to create 1 rle with new method : 0.014444828033447266 length of segment : 1008 time for calcul the mask position with numpy : 0.0006508827209472656 nb_pixel_total : 21613 time to create 1 rle with old method : 0.02407097816467285 length of segment : 230 time for calcul the mask position with numpy : 0.00035691261291503906 nb_pixel_total : 17079 time to create 1 rle with old method : 0.018881559371948242 length of segment : 158 time for calcul the mask position with numpy : 0.00015783309936523438 nb_pixel_total : 6785 time to create 1 rle with old method : 0.0077381134033203125 length of segment : 159 time for calcul the mask position with numpy : 0.0004074573516845703 nb_pixel_total : 10386 time to create 1 rle with old method : 0.011368751525878906 length of segment : 177 time for calcul the mask position with numpy : 0.0018012523651123047 nb_pixel_total : 102939 time to create 1 rle with old method : 0.11148452758789062 length of segment : 538 time for calcul the mask position with numpy : 0.0005877017974853516 nb_pixel_total : 21951 time to create 1 rle with old method : 0.02419424057006836 length of segment : 204 time for calcul the mask position with numpy : 0.00024008750915527344 nb_pixel_total : 6151 time to create 1 rle with old method : 0.006955385208129883 length of segment : 122 time for calcul the mask position with numpy : 0.00035452842712402344 nb_pixel_total : 6648 time to create 1 rle with old method : 0.007651805877685547 length of segment : 163 time spent for convertir_results : 3.5530223846435547 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 46 chid ids of type : 3594 Number RLEs to save : 10866 save missing photos in datou_result : time spend for datou_step_exec : 25.388864040374756 time spend to save output : 0.6488502025604248 total time spend for step 1 : 26.03771424293518 step2:crop_condition Thu Aug 21 10:40:56 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 13 ! batch 1 Loaded 46 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 15 About to insert : list_path_to_insert length 15 new photo from crops ! About to upload 15 photos upload in portfolio : 3736932 init cache_photo without model_param we have 15 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1755765658_739567 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 15 photos in the portfolio 3736932 time of upload the photos Elapsed time : 4.127491474151611 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 11 About to insert : list_path_to_insert length 11 new photo from crops ! About to upload 11 photos upload in portfolio : 3736932 init cache_photo without model_param we have 11 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1755765664_739567 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 11 photos in the portfolio 3736932 time of upload the photos Elapsed time : 3.050776720046997 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 19 About to insert : list_path_to_insert length 19 new photo from crops ! About to upload 19 photos upload in portfolio : 3736932 init cache_photo without model_param we have 19 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1755765675_739567 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 19 photos in the portfolio 3736932 time of upload the photos Elapsed time : 5.693636655807495 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1755765681_739567 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.6620044708251953 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1378197880, 1378197866, 1378197144, 1378197137, 1378197128, 1378197126, 1378197125, 1378197117, 1378197115, 1378197111, 1378197110, 1378197108, 1378196995] Looping around the photos to save general results len do output : 46 /1378201028Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201029Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201030Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201031Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201032Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201033Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201034Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201035Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201036Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201037Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201038Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201039Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201040Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201041Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201042Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201043Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201044Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201045Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201046Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201047Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201048Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201049Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201050Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201051Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201052Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201053Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201073Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201074Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201075Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201076Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201077Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201078Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201079Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201080Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201081Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201082Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201083Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201084Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201085Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201086Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201087Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201088Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201090Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201091Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201092Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378201093Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197880', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197866', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197144', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197137', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197128', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197126', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197125', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197117', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197115', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197111', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197110', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197108', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378196995', None, None, None, None, None, '3568032') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 151 time used for this insertion : 0.022446632385253906 save_final save missing photos in datou_result : time spend for datou_step_exec : 24.95113205909729 time spend to save output : 0.024461030960083008 total time spend for step 2 : 24.975593090057373 step3:rle_unique_nms_with_priority Thu Aug 21 10:41:21 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 46 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.4656858444213867 time for calcul the mask position with numpy : 0.6463437080383301 nb_pixel_total : 2053087 time to create 1 rle with new method : 0.07908177375793457 time for calcul the mask position with numpy : 0.006781816482543945 nb_pixel_total : 8658 time to create 1 rle with old method : 0.009706258773803711 time for calcul the mask position with numpy : 0.006034135818481445 nb_pixel_total : 11855 time to create 1 rle with old method : 0.01305389404296875 create new chi : 0.772362470626831 time to delete rle : 0.02679729461669922 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 1734 TO DO : save crop sub photo not yet done ! save time : 0.1438307762145996 nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 0.40325069427490234 time for calcul the mask position with numpy : 0.05423545837402344 nb_pixel_total : 2018569 time to create 1 rle with new method : 0.2294473648071289 time for calcul the mask position with numpy : 0.010804176330566406 nb_pixel_total : 7353 time to create 1 rle with old method : 0.008217811584472656 time for calcul the mask position with numpy : 0.010244607925415039 nb_pixel_total : 24832 time to create 1 rle with old method : 0.027309179306030273 time for calcul the mask position with numpy : 0.006135225296020508 nb_pixel_total : 12464 time to create 1 rle with old method : 0.013663053512573242 time for calcul the mask position with numpy : 0.006158351898193359 nb_pixel_total : 10382 time to create 1 rle with old method : 0.011553049087524414 create new chi : 0.3930549621582031 time to delete rle : 0.0003879070281982422 batch 1 Loaded 9 chid ids of type : 3594 ++++Number RLEs to save : 2178 TO DO : save crop sub photo not yet done ! save time : 0.16138768196105957 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.12975096702575684 time for calcul the mask position with numpy : 0.1011056900024414 nb_pixel_total : 1984039 time to create 1 rle with new method : 0.07337760925292969 time for calcul the mask position with numpy : 0.007113933563232422 nb_pixel_total : 3528 time to create 1 rle with old method : 0.005130290985107422 time for calcul the mask position with numpy : 0.007924079895019531 nb_pixel_total : 76044 time to create 1 rle with old method : 0.08821225166320801 time for calcul the mask position with numpy : 0.0064051151275634766 nb_pixel_total : 9989 time to create 1 rle with old method : 0.01124715805053711 create new chi : 0.3106245994567871 time to delete rle : 0.0004093647003173828 batch 1 Loaded 7 chid ids of type : 3594 +++Number RLEs to save : 2378 TO DO : save crop sub photo not yet done ! save time : 0.1620182991027832 nb_obj : 4 nb_hashtags : 3 time to prepare the origin masks : 0.10970640182495117 time for calcul the mask position with numpy : 0.10567975044250488 nb_pixel_total : 2026012 time to create 1 rle with new method : 0.09230756759643555 time for calcul the mask position with numpy : 0.007253170013427734 nb_pixel_total : 8177 time to create 1 rle with old method : 0.009719371795654297 time for calcul the mask position with numpy : 0.0075342655181884766 nb_pixel_total : 15551 time to create 1 rle with old method : 0.017723798751831055 time for calcul the mask position with numpy : 0.006994485855102539 nb_pixel_total : 4066 time to create 1 rle with old method : 0.0047075748443603516 time for calcul the mask position with numpy : 0.0070111751556396484 nb_pixel_total : 19794 time to create 1 rle with old method : 0.023193359375 create new chi : 0.29436302185058594 time to delete rle : 0.0004436969757080078 batch 1 Loaded 9 chid ids of type : 3594 +++++Number RLEs to save : 2076 TO DO : save crop sub photo not yet done ! save time : 0.15712261199951172 nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 0.07144451141357422 time for calcul the mask position with numpy : 0.20937657356262207 nb_pixel_total : 1210459 time to create 1 rle with new method : 0.26587915420532227 time for calcul the mask position with numpy : 0.011053323745727539 nb_pixel_total : 718501 time to create 1 rle with new method : 0.07646489143371582 time for calcul the mask position with numpy : 0.006854534149169922 nb_pixel_total : 123802 time to create 1 rle with old method : 0.15392112731933594 time for calcul the mask position with numpy : 0.006578207015991211 nb_pixel_total : 3710 time to create 1 rle with old method : 0.004182100296020508 time for calcul the mask position with numpy : 0.006466865539550781 nb_pixel_total : 17128 time to create 1 rle with old method : 0.019291400909423828 create new chi : 0.7751715183258057 time to delete rle : 0.0007579326629638672 batch 1 Loaded 9 chid ids of type : 3594 ++++Number RLEs to save : 4603 TO DO : save crop sub photo not yet done ! save time : 0.29048871994018555 nb_obj : 10 nb_hashtags : 3 time to prepare the origin masks : 0.5082948207855225 time for calcul the mask position with numpy : 0.18535208702087402 nb_pixel_total : 1916876 time to create 1 rle with new method : 0.607468843460083 time for calcul the mask position with numpy : 0.007124423980712891 nb_pixel_total : 5902 time to create 1 rle with old method : 0.009653091430664062 time for calcul the mask position with numpy : 0.006600141525268555 nb_pixel_total : 11306 time to create 1 rle with old method : 0.018817663192749023 time for calcul the mask position with numpy : 0.006703615188598633 nb_pixel_total : 14468 time to create 1 rle with old method : 0.02356576919555664 time for calcul the mask position with numpy : 0.006208181381225586 nb_pixel_total : 9094 time to create 1 rle with old method : 0.010200262069702148 time for calcul the mask position with numpy : 0.006487369537353516 nb_pixel_total : 1548 time to create 1 rle with old method : 0.0018241405487060547 time for calcul the mask position with numpy : 0.006677150726318359 nb_pixel_total : 3348 time to create 1 rle with old method : 0.0037140846252441406 time for calcul the mask position with numpy : 0.006167173385620117 nb_pixel_total : 1436 time to create 1 rle with old method : 0.0015645027160644531 time for calcul the mask position with numpy : 0.006714820861816406 nb_pixel_total : 11080 time to create 1 rle with old method : 0.01212620735168457 time for calcul the mask position with numpy : 0.006642341613769531 nb_pixel_total : 2498 time to create 1 rle with old method : 0.002880096435546875 time for calcul the mask position with numpy : 0.006780862808227539 nb_pixel_total : 96044 time to create 1 rle with old method : 0.10511136054992676 create new chi : 1.0621616840362549 time to delete rle : 0.0006680488586425781 batch 1 Loaded 21 chid ids of type : 3594 ++++++++++++++Number RLEs to save : 3851 TO DO : save crop sub photo not yet done ! save time : 0.24750995635986328 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.042584896087646484 time for calcul the mask position with numpy : 0.027422666549682617 nb_pixel_total : 2051034 time to create 1 rle with new method : 0.0332643985748291 time for calcul the mask position with numpy : 0.0060901641845703125 nb_pixel_total : 8919 time to create 1 rle with old method : 0.009992122650146484 time for calcul the mask position with numpy : 0.005995035171508789 nb_pixel_total : 13647 time to create 1 rle with old method : 0.01517033576965332 create new chi : 0.09819650650024414 time to delete rle : 0.0003077983856201172 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 1536 TO DO : save crop sub photo not yet done ! save time : 0.12937712669372559 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.057909250259399414 time for calcul the mask position with numpy : 0.01635003089904785 nb_pixel_total : 1354727 time to create 1 rle with new method : 0.031847476959228516 time for calcul the mask position with numpy : 0.006588459014892578 nb_pixel_total : 71586 time to create 1 rle with old method : 0.08076620101928711 time for calcul the mask position with numpy : 0.011838912963867188 nb_pixel_total : 642568 time to create 1 rle with new method : 0.032294511795043945 time for calcul the mask position with numpy : 0.006558656692504883 nb_pixel_total : 4719 time to create 1 rle with old method : 0.005318880081176758 create new chi : 0.19219970703125 time to delete rle : 0.0006825923919677734 batch 1 Loaded 7 chid ids of type : 3594 +++Number RLEs to save : 4272 TO DO : save crop sub photo not yet done ! save time : 0.2626357078552246 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.4604027271270752 time for calcul the mask position with numpy : 0.055757999420166016 nb_pixel_total : 1959186 time to create 1 rle with new method : 0.24826979637145996 time for calcul the mask position with numpy : 0.0067789554595947266 nb_pixel_total : 78590 time to create 1 rle with old method : 0.08818840980529785 time for calcul the mask position with numpy : 0.0065691471099853516 nb_pixel_total : 7152 time to create 1 rle with old method : 0.008061885833740234 time for calcul the mask position with numpy : 0.006238698959350586 nb_pixel_total : 28672 time to create 1 rle with old method : 0.03183269500732422 create new chi : 0.46274805068969727 time to delete rle : 0.00042819976806640625 batch 1 Loaded 7 chid ids of type : 3594 ++++Number RLEs to save : 2670 TO DO : save crop sub photo not yet done ! save time : 0.18837308883666992 nb_obj : 3 nb_hashtags : 3 time to prepare the origin masks : 0.20585393905639648 time for calcul the mask position with numpy : 0.05340862274169922 nb_pixel_total : 1853759 time to create 1 rle with new method : 0.1979660987854004 time for calcul the mask position with numpy : 0.007582187652587891 nb_pixel_total : 189393 time to create 1 rle with new method : 0.22857880592346191 time for calcul the mask position with numpy : 0.007227182388305664 nb_pixel_total : 25515 time to create 1 rle with old method : 0.03180265426635742 time for calcul the mask position with numpy : 0.006201982498168945 nb_pixel_total : 4933 time to create 1 rle with old method : 0.005549192428588867 create new chi : 0.5490450859069824 time to delete rle : 0.0005970001220703125 batch 1 Loaded 7 chid ids of type : 3594 +++++++Number RLEs to save : 3644 TO DO : save crop sub photo not yet done ! save time : 0.24215960502624512 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.05147552490234375 time for calcul the mask position with numpy : 0.08352255821228027 nb_pixel_total : 2051987 time to create 1 rle with new method : 0.46547722816467285 time for calcul the mask position with numpy : 0.010874032974243164 nb_pixel_total : 21613 time to create 1 rle with old method : 0.02437567710876465 create new chi : 0.5930051803588867 time to delete rle : 0.0002868175506591797 batch 1 Loaded 3 chid ids of type : 3594 ++Number RLEs to save : 1540 TO DO : save crop sub photo not yet done ! save time : 0.12865495681762695 nb_obj : 4 nb_hashtags : 3 time to prepare the origin masks : 0.6857948303222656 time for calcul the mask position with numpy : 0.37471461296081543 nb_pixel_total : 1936411 time to create 1 rle with new method : 0.214141845703125 time for calcul the mask position with numpy : 0.006833314895629883 nb_pixel_total : 102939 time to create 1 rle with old method : 0.1126546859741211 time for calcul the mask position with numpy : 0.0059452056884765625 nb_pixel_total : 10386 time to create 1 rle with old method : 0.011049032211303711 time for calcul the mask position with numpy : 0.005901813507080078 nb_pixel_total : 6785 time to create 1 rle with old method : 0.007069587707519531 time for calcul the mask position with numpy : 0.006105184555053711 nb_pixel_total : 17079 time to create 1 rle with old method : 0.018296480178833008 create new chi : 0.7729558944702148 time to delete rle : 0.0005154609680175781 batch 1 Loaded 9 chid ids of type : 3594 ++++Number RLEs to save : 3144 TO DO : save crop sub photo not yet done ! save time : 0.2091825008392334 nb_obj : 3 nb_hashtags : 1 time to prepare the origin masks : 0.051206350326538086 time for calcul the mask position with numpy : 0.019231557846069336 nb_pixel_total : 2038850 time to create 1 rle with new method : 0.23441410064697266 time for calcul the mask position with numpy : 0.00622868537902832 nb_pixel_total : 6648 time to create 1 rle with old method : 0.0073430538177490234 time for calcul the mask position with numpy : 0.006127357482910156 nb_pixel_total : 6151 time to create 1 rle with old method : 0.0067272186279296875 time for calcul the mask position with numpy : 0.0061817169189453125 nb_pixel_total : 21951 time to create 1 rle with old method : 0.023707866668701172 create new chi : 0.3103156089782715 time to delete rle : 0.0003387928009033203 batch 1 Loaded 7 chid ids of type : 3594 ++++Number RLEs to save : 2058 TO DO : save crop sub photo not yet done ! save time : 0.14424467086791992 map_output_result : {1378197880: (0.0, 'Should be the crop_list due to order', 0), 1378197866: (0.0, 'Should be the crop_list due to order', 0), 1378197144: (0.0, 'Should be the crop_list due to order', 0), 1378197137: (0.0, 'Should be the crop_list due to order', 0), 1378197128: (0.0, 'Should be the crop_list due to order', 0), 1378197126: (0.0, 'Should be the crop_list due to order', 0), 1378197125: (0.0, 'Should be the crop_list due to order', 0), 1378197117: (0.0, 'Should be the crop_list due to order', 0), 1378197115: (0.0, 'Should be the crop_list due to order', 0), 1378197111: (0.0, 'Should be the crop_list due to order', 0), 1378197110: (0.0, 'Should be the crop_list due to order', 0), 1378197108: (0.0, 'Should be the crop_list due to order', 0), 1378196995: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1378197880, 1378197866, 1378197144, 1378197137, 1378197128, 1378197126, 1378197125, 1378197117, 1378197115, 1378197111, 1378197110, 1378197108, 1378196995] Looping around the photos to save general results len do output : 13 /1378197880.Didn't retrieve data . /1378197866.Didn't retrieve data . /1378197144.Didn't retrieve data . /1378197137.Didn't retrieve data . /1378197128.Didn't retrieve data . /1378197126.Didn't retrieve data . /1378197125.Didn't retrieve data . /1378197117.Didn't retrieve data . /1378197115.Didn't retrieve data . /1378197111.Didn't retrieve data . /1378197110.Didn't retrieve data . /1378197108.Didn't retrieve data . /1378196995.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197880', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197866', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197144', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197137', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197128', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197126', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197125', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197117', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197115', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197111', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197110', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197108', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378196995', None, None, None, None, None, '3568032') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 39 time used for this insertion : 0.013585329055786133 save_final save missing photos in datou_result : time spend for datou_step_exec : 12.849670171737671 time spend to save output : 0.01414179801940918 total time spend for step 3 : 12.86381196975708 step4:ventilate_hashtags_in_portfolio Thu Aug 21 10:41:34 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 26116300 get user id for portfolio 26116300 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26116300 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('environnement','pehd','pet_clair','papier','autre','pet_fonce','mal_croppe','metal','carton','background','flou')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26116300 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('environnement','pehd','pet_clair','papier','autre','pet_fonce','mal_croppe','metal','carton','background','flou')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26116300 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('environnement','pehd','pet_clair','papier','autre','pet_fonce','mal_croppe','metal','carton','background','flou')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/26116499,26116500,26116501,26116502,26116503,26116504,26116505,26116506,26116507,26116508,26116509?tags=environnement,pehd,pet_clair,papier,autre,pet_fonce,mal_croppe,metal,carton,background,flou Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1378197880, 1378197866, 1378197144, 1378197137, 1378197128, 1378197126, 1378197125, 1378197117, 1378197115, 1378197111, 1378197110, 1378197108, 1378196995] Looping around the photos to save general results len do output : 1 /26116300. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197880', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197866', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197144', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197137', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197128', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197126', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197125', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197117', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197115', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197111', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197110', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197108', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378196995', None, None, None, None, None, '3568032') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 14 time used for this insertion : 0.01614522933959961 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.6402561664581299 time spend to save output : 0.016434669494628906 total time spend for step 4 : 1.6566908359527588 step5:final Thu Aug 21 10:41:36 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1378197880: ('0.09280790004748335',), 1378197866: ('0.09280790004748335',), 1378197144: ('0.09280790004748335',), 1378197137: ('0.09280790004748335',), 1378197128: ('0.09280790004748335',), 1378197126: ('0.09280790004748335',), 1378197125: ('0.09280790004748335',), 1378197117: ('0.09280790004748335',), 1378197115: ('0.09280790004748335',), 1378197111: ('0.09280790004748335',), 1378197110: ('0.09280790004748335',), 1378197108: ('0.09280790004748335',), 1378196995: ('0.09280790004748335',)} new output for save of step final : {1378197880: ('0.09280790004748335',), 1378197866: ('0.09280790004748335',), 1378197144: ('0.09280790004748335',), 1378197137: ('0.09280790004748335',), 1378197128: ('0.09280790004748335',), 1378197126: ('0.09280790004748335',), 1378197125: ('0.09280790004748335',), 1378197117: ('0.09280790004748335',), 1378197115: ('0.09280790004748335',), 1378197111: ('0.09280790004748335',), 1378197110: ('0.09280790004748335',), 1378197108: ('0.09280790004748335',), 1378196995: ('0.09280790004748335',)} [1378197880, 1378197866, 1378197144, 1378197137, 1378197128, 1378197126, 1378197125, 1378197117, 1378197115, 1378197111, 1378197110, 1378197108, 1378196995] Looping around the photos to save general results len do output : 13 /1378197880.Didn't retrieve data . /1378197866.Didn't retrieve data . /1378197144.Didn't retrieve data . /1378197137.Didn't retrieve data . /1378197128.Didn't retrieve data . /1378197126.Didn't retrieve data . /1378197125.Didn't retrieve data . /1378197117.Didn't retrieve data . /1378197115.Didn't retrieve data . /1378197111.Didn't retrieve data . /1378197110.Didn't retrieve data . /1378197108.Didn't retrieve data . /1378196995.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197880', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197866', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197144', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197137', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197128', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197126', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197125', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197117', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197115', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197111', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197110', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197108', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378196995', None, None, None, None, None, '3568032') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 39 time used for this insertion : 0.01328277587890625 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.12534689903259277 time spend to save output : 0.013889789581298828 total time spend for step 5 : 0.1392366886138916 step6:blur_detection Thu Aug 21 10:41:36 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1755765629_739567_1378197880_765f2391a69fdb18e4ce78fbe18e7453.jpg resize: (1080, 1920) 1378197880 0.23485078820123934 treat image : temp/1755765629_739567_1378197866_a4af216e054d2f4438bf936be06676ac.jpg resize: (1080, 1920) 1378197866 -0.03908041854672473 treat image : temp/1755765629_739567_1378197144_539cd90f6c67556efac4d5574a8c06f5.jpg resize: (1080, 1920) 1378197144 -0.1361337917662602 treat image : temp/1755765629_739567_1378197137_22edb441b82e24db373b67af5307951f.jpg resize: (1080, 1920) 1378197137 -3.833650473987227 treat image : temp/1755765629_739567_1378197128_998f087d677e611104583fa800b688d1.jpg resize: (1080, 1920) 1378197128 -0.4397765120800879 treat image : temp/1755765629_739567_1378197126_6da2c0f373e7a524c4e638ab78c6b2f8.jpg resize: (1080, 1920) 1378197126 -3.4052492975812236 treat image : temp/1755765629_739567_1378197125_ac791edba8967964a741f49ed5c5de66.jpg resize: (1080, 1920) 1378197125 -3.7200072263269424 treat image : temp/1755765629_739567_1378197117_cfcecad2b4fb5b875b579301e0f80cf1.jpg resize: (1080, 1920) 1378197117 -0.913838968273692 treat image : temp/1755765629_739567_1378197115_1143cc63dab660f442af7c520d7c1bcd.jpg resize: (1080, 1920) 1378197115 -0.08157528508458024 treat image : temp/1755765629_739567_1378197111_6fe06123fd7420903033444ec75c0f89.jpg resize: (1080, 1920) 1378197111 3.7357849815636213 treat image : temp/1755765629_739567_1378197110_61ac59f8832ae26761e2a47636ec21f9.jpg resize: (1080, 1920) 1378197110 0.6604842255286256 treat image : temp/1755765629_739567_1378197108_ff589e3ed822aa1b4aa412cffd22bf1c.jpg resize: (1080, 1920) 1378197108 -4.268551188126451 treat image : temp/1755765629_739567_1378196995_5267a679be55e78f1504b3d079a64337.jpg resize: (1080, 1920) 1378196995 -0.5531050968037737 treat image : temp/1755765629_739567_1378197880_765f2391a69fdb18e4ce78fbe18e7453_rle_crop_3922746862_0.png resize: (172, 96) 1378201028 -0.7386188330038561 treat image : temp/1755765629_739567_1378197866_a4af216e054d2f4438bf936be06676ac_rle_crop_3922746863_0.png resize: (148, 213) 1378201029 -1.451016816941892 treat image : temp/1755765629_739567_1378197866_a4af216e054d2f4438bf936be06676ac_rle_crop_3922746866_0.png resize: (79, 124) 1378201030 -1.186929929573404 treat image : temp/1755765629_739567_1378197144_539cd90f6c67556efac4d5574a8c06f5_rle_crop_3922746867_0.png resize: (173, 106) 1378201031 -0.4614618484766752 treat image : temp/1755765629_739567_1378197137_22edb441b82e24db373b67af5307951f_rle_crop_3922746871_0.png resize: (61, 91) 1378201032 0.46311577144474636 treat image : temp/1755765629_739567_1378197137_22edb441b82e24db373b67af5307951f_rle_crop_3922746870_0.png resize: (142, 215) 1378201033 -2.5257639747657876 treat image : temp/1755765629_739567_1378197128_998f087d677e611104583fa800b688d1_rle_crop_3922746875_0.png resize: (84, 62) 1378201034 1.6691414328067393 treat image : temp/1755765629_739567_1378197126_6da2c0f373e7a524c4e638ab78c6b2f8_rle_crop_3922746880_0.png resize: (122, 134) 1378201035 -5.034807511941003 treat image : temp/1755765629_739567_1378197125_ac791edba8967964a741f49ed5c5de66_rle_crop_3922746889_0.png resize: (121, 150) 1378201036 -4.1885744202099495 treat image : temp/1755765629_739567_1378197117_cfcecad2b4fb5b875b579301e0f80cf1_rle_crop_3922746890_0.png resize: (101, 76) 1378201037 -1.4229532991940885 treat image : temp/1755765629_739567_1378197111_6fe06123fd7420903033444ec75c0f89_rle_crop_3922746896_0.png resize: (111, 85) 1378201038 -1.1301739368257258 treat image : temp/1755765629_739567_1378197108_ff589e3ed822aa1b4aa412cffd22bf1c_rle_crop_3922746902_0.png resize: (177, 105) 1378201039 -0.6318024389952502 treat image : temp/1755765629_739567_1378196995_5267a679be55e78f1504b3d079a64337_rle_crop_3922746906_0.png resize: (163, 74) 1378201040 -1.1745633785955079 treat image : temp/1755765629_739567_1378196995_5267a679be55e78f1504b3d079a64337_rle_crop_3922746904_0.png resize: (198, 185) 1378201041 -1.1808368703733976 treat image : temp/1755765629_739567_1378196995_5267a679be55e78f1504b3d079a64337_rle_crop_3922746905_0.png resize: (122, 63) 1378201042 -0.0037000228876736983 treat image : temp/1755765629_739567_1378197137_22edb441b82e24db373b67af5307951f_rle_crop_3922746873_0.png resize: (141, 74) 1378201043 -0.10767140299143883 treat image : temp/1755765629_739567_1378197126_6da2c0f373e7a524c4e638ab78c6b2f8_rle_crop_3922746885_0.png resize: (108, 166) 1378201044 -2.211526668464176 treat image : temp/1755765629_739567_1378197126_6da2c0f373e7a524c4e638ab78c6b2f8_rle_crop_3922746886_0.png resize: (174, 134) 1378201045 -4.130811980671152 treat image : temp/1755765629_739567_1378197126_6da2c0f373e7a524c4e638ab78c6b2f8_rle_crop_3922746879_0.png resize: (53, 89) 1378201046 -3.363626032684245 treat image : temp/1755765629_739567_1378197126_6da2c0f373e7a524c4e638ab78c6b2f8_rle_crop_3922746887_0.png resize: (121, 68) 1378201047 -2.3033635456894377 treat image : temp/1755765629_739567_1378197126_6da2c0f373e7a524c4e638ab78c6b2f8_rle_crop_3922746883_0.png resize: (56, 54) 1378201048 -1.0964634439389684 treat image : temp/1755765629_739567_1378197126_6da2c0f373e7a524c4e638ab78c6b2f8_rle_crop_3922746881_0.png resize: (50, 38) 1378201049 -4.819574934673999 treat image : temp/1755765629_739567_1378197126_6da2c0f373e7a524c4e638ab78c6b2f8_rle_crop_3922746882_0.png resize: (67, 71) 1378201050 -0.523367855949425 treat image : temp/1755765629_739567_1378197126_6da2c0f373e7a524c4e638ab78c6b2f8_rle_crop_3922746878_0.png resize: (363, 588) 1378201051 -1.2844055273638102 treat image : temp/1755765629_739567_1378197115_1143cc63dab660f442af7c520d7c1bcd_rle_crop_3922746893_0.png resize: (204, 248) 1378201052 0.2331165697235134 treat image : temp/1755765629_739567_1378197111_6fe06123fd7420903033444ec75c0f89_rle_crop_3922746898_0.png resize: (696, 696) 1378201053 -0.5932034954801284 treat image : temp/1755765629_739567_1378197880_765f2391a69fdb18e4ce78fbe18e7453_rle_crop_3922746861_0.png resize: (154, 101) 1378201073 -0.8880982659440141 treat image : temp/1755765629_739567_1378197866_a4af216e054d2f4438bf936be06676ac_rle_crop_3922746865_0.png resize: (199, 165) 1378201074 -1.8337597620342638 treat image : temp/1755765629_739567_1378197866_a4af216e054d2f4438bf936be06676ac_rle_crop_3922746864_0.png resize: (119, 140) 1378201075 -0.4394493363757277 treat image : temp/1755765629_739567_1378197144_539cd90f6c67556efac4d5574a8c06f5_rle_crop_3922746868_0.png resize: (428, 311) 1378201076 -0.21610881002916196 treat image : temp/1755765629_739567_1378197144_539cd90f6c67556efac4d5574a8c06f5_rle_crop_3922746869_0.png resize: (42, 98) 1378201077 -1.0531808207708373 treat image : temp/1755765629_739567_1378197137_22edb441b82e24db373b67af5307951f_rle_crop_3922746872_0.png resize: (126, 149) 1378201078 -3.6912797684068783 treat image : temp/1755765629_739567_1378197128_998f087d677e611104583fa800b688d1_rle_crop_3922746877_0.png resize: (957, 1056) 1378201079 0.2594722910844673 treat image : temp/1755765629_739567_1378197128_998f087d677e611104583fa800b688d1_rle_crop_3922746874_0.png resize: (146, 144) 1378201080 -2.398879946349803 treat image : temp/1755765629_739567_1378197128_998f087d677e611104583fa800b688d1_rle_crop_3922746876_0.png resize: (572, 364) 1378201081 -0.47777604658223055 treat image : temp/1755765629_739567_1378197126_6da2c0f373e7a524c4e638ab78c6b2f8_rle_crop_3922746884_0.png resize: (98, 139) 1378201082 -4.205174261016169 treat image : temp/1755765629_739567_1378197125_ac791edba8967964a741f49ed5c5de66_rle_crop_3922746888_0.png resize: (104, 170) 1378201083 -0.06860615682008264 treat image : temp/1755765629_739567_1378197117_cfcecad2b4fb5b875b579301e0f80cf1_rle_crop_3922746891_0.png resize: (941, 982) 1378201084 0.3591120225884661 treat image : temp/1755765629_739567_1378197117_cfcecad2b4fb5b875b579301e0f80cf1_rle_crop_3922746892_0.png resize: (459, 304) 1378201085 -0.1233845904881829 treat image : temp/1755765629_739567_1378197115_1143cc63dab660f442af7c520d7c1bcd_rle_crop_3922746894_0.png resize: (80, 121) 1378201086 -1.6799875205526131 treat image : temp/1755765629_739567_1378197115_1143cc63dab660f442af7c520d7c1bcd_rle_crop_3922746895_0.png resize: (493, 289) 1378201087 0.245824587908098 treat image : temp/1755765629_739567_1378197111_6fe06123fd7420903033444ec75c0f89_rle_crop_3922746897_0.png resize: (157, 294) 1378201088 -1.8205768553333044 treat image : temp/1755765629_739567_1378197110_61ac59f8832ae26761e2a47636ec21f9_rle_crop_3922746899_0.png resize: (197, 177) 1378201090 -1.9189127960941326 treat image : temp/1755765629_739567_1378197108_ff589e3ed822aa1b4aa412cffd22bf1c_rle_crop_3922746900_0.png resize: (140, 144) 1378201091 -3.4421489929770157 treat image : temp/1755765629_739567_1378197108_ff589e3ed822aa1b4aa412cffd22bf1c_rle_crop_3922746903_0.png resize: (534, 333) 1378201092 0.38223490893837014 treat image : temp/1755765629_739567_1378197108_ff589e3ed822aa1b4aa412cffd22bf1c_rle_crop_3922746901_0.png resize: (158, 57) 1378201093 -2.381097548652733 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 59 time used for this insertion : 0.012584924697875977 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 59 time used for this insertion : 0.01288151741027832 save missing photos in datou_result : time spend for datou_step_exec : 11.687505960464478 time spend to save output : 0.03050088882446289 total time spend for step 6 : 11.71800684928894 step7:brightness Thu Aug 21 10:41:48 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1755765629_739567_1378197880_765f2391a69fdb18e4ce78fbe18e7453.jpg treat image : temp/1755765629_739567_1378197866_a4af216e054d2f4438bf936be06676ac.jpg treat image : temp/1755765629_739567_1378197144_539cd90f6c67556efac4d5574a8c06f5.jpg treat image : temp/1755765629_739567_1378197137_22edb441b82e24db373b67af5307951f.jpg treat image : temp/1755765629_739567_1378197128_998f087d677e611104583fa800b688d1.jpg treat image : temp/1755765629_739567_1378197126_6da2c0f373e7a524c4e638ab78c6b2f8.jpg treat image : temp/1755765629_739567_1378197125_ac791edba8967964a741f49ed5c5de66.jpg treat image : temp/1755765629_739567_1378197117_cfcecad2b4fb5b875b579301e0f80cf1.jpg treat image : temp/1755765629_739567_1378197115_1143cc63dab660f442af7c520d7c1bcd.jpg treat image : temp/1755765629_739567_1378197111_6fe06123fd7420903033444ec75c0f89.jpg treat image : temp/1755765629_739567_1378197110_61ac59f8832ae26761e2a47636ec21f9.jpg treat image : temp/1755765629_739567_1378197108_ff589e3ed822aa1b4aa412cffd22bf1c.jpg treat image : temp/1755765629_739567_1378196995_5267a679be55e78f1504b3d079a64337.jpg treat image : temp/1755765629_739567_1378197880_765f2391a69fdb18e4ce78fbe18e7453_rle_crop_3922746862_0.png treat image : temp/1755765629_739567_1378197866_a4af216e054d2f4438bf936be06676ac_rle_crop_3922746863_0.png treat image : temp/1755765629_739567_1378197866_a4af216e054d2f4438bf936be06676ac_rle_crop_3922746866_0.png treat image : temp/1755765629_739567_1378197144_539cd90f6c67556efac4d5574a8c06f5_rle_crop_3922746867_0.png treat image : temp/1755765629_739567_1378197137_22edb441b82e24db373b67af5307951f_rle_crop_3922746871_0.png treat image : temp/1755765629_739567_1378197137_22edb441b82e24db373b67af5307951f_rle_crop_3922746870_0.png treat image : temp/1755765629_739567_1378197128_998f087d677e611104583fa800b688d1_rle_crop_3922746875_0.png treat image : temp/1755765629_739567_1378197126_6da2c0f373e7a524c4e638ab78c6b2f8_rle_crop_3922746880_0.png treat image : temp/1755765629_739567_1378197125_ac791edba8967964a741f49ed5c5de66_rle_crop_3922746889_0.png treat image : temp/1755765629_739567_1378197117_cfcecad2b4fb5b875b579301e0f80cf1_rle_crop_3922746890_0.png treat image : temp/1755765629_739567_1378197111_6fe06123fd7420903033444ec75c0f89_rle_crop_3922746896_0.png treat image : temp/1755765629_739567_1378197108_ff589e3ed822aa1b4aa412cffd22bf1c_rle_crop_3922746902_0.png treat image : temp/1755765629_739567_1378196995_5267a679be55e78f1504b3d079a64337_rle_crop_3922746906_0.png treat image : temp/1755765629_739567_1378196995_5267a679be55e78f1504b3d079a64337_rle_crop_3922746904_0.png treat image : temp/1755765629_739567_1378196995_5267a679be55e78f1504b3d079a64337_rle_crop_3922746905_0.png treat image : temp/1755765629_739567_1378197137_22edb441b82e24db373b67af5307951f_rle_crop_3922746873_0.png treat image : temp/1755765629_739567_1378197126_6da2c0f373e7a524c4e638ab78c6b2f8_rle_crop_3922746885_0.png treat image : temp/1755765629_739567_1378197126_6da2c0f373e7a524c4e638ab78c6b2f8_rle_crop_3922746886_0.png treat image : temp/1755765629_739567_1378197126_6da2c0f373e7a524c4e638ab78c6b2f8_rle_crop_3922746879_0.png treat image : temp/1755765629_739567_1378197126_6da2c0f373e7a524c4e638ab78c6b2f8_rle_crop_3922746887_0.png treat image : temp/1755765629_739567_1378197126_6da2c0f373e7a524c4e638ab78c6b2f8_rle_crop_3922746883_0.png treat image : temp/1755765629_739567_1378197126_6da2c0f373e7a524c4e638ab78c6b2f8_rle_crop_3922746881_0.png treat image : temp/1755765629_739567_1378197126_6da2c0f373e7a524c4e638ab78c6b2f8_rle_crop_3922746882_0.png treat image : temp/1755765629_739567_1378197126_6da2c0f373e7a524c4e638ab78c6b2f8_rle_crop_3922746878_0.png treat image : temp/1755765629_739567_1378197115_1143cc63dab660f442af7c520d7c1bcd_rle_crop_3922746893_0.png treat image : temp/1755765629_739567_1378197111_6fe06123fd7420903033444ec75c0f89_rle_crop_3922746898_0.png treat image : temp/1755765629_739567_1378197880_765f2391a69fdb18e4ce78fbe18e7453_rle_crop_3922746861_0.png treat image : temp/1755765629_739567_1378197866_a4af216e054d2f4438bf936be06676ac_rle_crop_3922746865_0.png treat image : temp/1755765629_739567_1378197866_a4af216e054d2f4438bf936be06676ac_rle_crop_3922746864_0.png treat image : temp/1755765629_739567_1378197144_539cd90f6c67556efac4d5574a8c06f5_rle_crop_3922746868_0.png treat image : temp/1755765629_739567_1378197144_539cd90f6c67556efac4d5574a8c06f5_rle_crop_3922746869_0.png treat image : temp/1755765629_739567_1378197137_22edb441b82e24db373b67af5307951f_rle_crop_3922746872_0.png treat image : temp/1755765629_739567_1378197128_998f087d677e611104583fa800b688d1_rle_crop_3922746877_0.png treat image : temp/1755765629_739567_1378197128_998f087d677e611104583fa800b688d1_rle_crop_3922746874_0.png treat image : temp/1755765629_739567_1378197128_998f087d677e611104583fa800b688d1_rle_crop_3922746876_0.png treat image : temp/1755765629_739567_1378197126_6da2c0f373e7a524c4e638ab78c6b2f8_rle_crop_3922746884_0.png treat image : temp/1755765629_739567_1378197125_ac791edba8967964a741f49ed5c5de66_rle_crop_3922746888_0.png treat image : temp/1755765629_739567_1378197117_cfcecad2b4fb5b875b579301e0f80cf1_rle_crop_3922746891_0.png treat image : temp/1755765629_739567_1378197117_cfcecad2b4fb5b875b579301e0f80cf1_rle_crop_3922746892_0.png treat image : temp/1755765629_739567_1378197115_1143cc63dab660f442af7c520d7c1bcd_rle_crop_3922746894_0.png treat image : temp/1755765629_739567_1378197115_1143cc63dab660f442af7c520d7c1bcd_rle_crop_3922746895_0.png treat image : temp/1755765629_739567_1378197111_6fe06123fd7420903033444ec75c0f89_rle_crop_3922746897_0.png treat image : temp/1755765629_739567_1378197110_61ac59f8832ae26761e2a47636ec21f9_rle_crop_3922746899_0.png treat image : temp/1755765629_739567_1378197108_ff589e3ed822aa1b4aa412cffd22bf1c_rle_crop_3922746900_0.png treat image : temp/1755765629_739567_1378197108_ff589e3ed822aa1b4aa412cffd22bf1c_rle_crop_3922746903_0.png treat image : temp/1755765629_739567_1378197108_ff589e3ed822aa1b4aa412cffd22bf1c_rle_crop_3922746901_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 59 time used for this insertion : 0.013550281524658203 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 59 time used for this insertion : 0.014938831329345703 save missing photos in datou_result : time spend for datou_step_exec : 3.2834599018096924 time spend to save output : 0.03349614143371582 total time spend for step 7 : 3.316956043243408 step8:velours_tree Thu Aug 21 10:41:51 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.08878517150878906 time spend to save output : 5.555152893066406e-05 total time spend for step 8 : 0.08884072303771973 step9:send_mail_cod Thu Aug 21 10:41:51 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P26116300_21-08-2025_10_41_51.pdf 26116500 imagette261165001755765711 26116501 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette261165011755765711 26116502 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette261165021755765713 26116503 change filename to text .imagette261165031755765713 26116504 imagette261165041755765713 26116505 imagette261165051755765713 26116506 imagette261165061755765714 26116507 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette261165071755765714 26116508 imagette261165081755765714 26116509 imagette261165091755765714 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=26116300 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/26116499,26116500,26116501,26116502,26116503,26116504,26116505,26116506,26116507,26116508,26116509?tags=environnement,pehd,pet_clair,papier,autre,pet_fonce,mal_croppe,metal,carton,background,flou args[1378197880] : ((1378197880, 0.23485078820123934, 492688767), (1378197880, 0.66176990900383, 2107752395), '0.09280790004748335') We are sending mail with results at report@fotonower.com args[1378197866] : ((1378197866, -0.03908041854672473, 492688767), (1378197866, 0.5969036489494207, 2107752395), '0.09280790004748335') We are sending mail with results at report@fotonower.com args[1378197144] : ((1378197144, -0.1361337917662602, 492688767), (1378197144, 0.721004964336528, 2107752395), '0.09280790004748335') We are sending mail with results at report@fotonower.com args[1378197137] : ((1378197137, -3.833650473987227, 492609224), (1378197137, 0.44870938159355295, 2107752395), '0.09280790004748335') We are sending mail with results at report@fotonower.com args[1378197128] : ((1378197128, -0.4397765120800879, 492688767), (1378197128, 0.29377717835253664, 2107752395), '0.09280790004748335') We are sending mail with results at report@fotonower.com args[1378197126] : ((1378197126, -3.4052492975812236, 492609224), (1378197126, 0.6273343257507891, 2107752395), '0.09280790004748335') We are sending mail with results at report@fotonower.com args[1378197125] : ((1378197125, -3.7200072263269424, 492609224), (1378197125, 0.5513349028340143, 2107752395), '0.09280790004748335') We are sending mail with results at report@fotonower.com args[1378197117] : ((1378197117, -0.913838968273692, 492688767), (1378197117, 1.3267895857242575, 2107752395), '0.09280790004748335') We are sending mail with results at report@fotonower.com args[1378197115] : ((1378197115, -0.08157528508458024, 492688767), (1378197115, 0.7400440978729931, 2107752395), '0.09280790004748335') We are sending mail with results at report@fotonower.com args[1378197111] : ((1378197111, 3.7357849815636213, 492688767), (1378197111, 0.3561312859929122, 2107752395), '0.09280790004748335') We are sending mail with results at report@fotonower.com args[1378197110] : ((1378197110, 0.6604842255286256, 492688767), (1378197110, 0.8608435920618348, 2107752395), '0.09280790004748335') We are sending mail with results at report@fotonower.com args[1378197108] : ((1378197108, -4.268551188126451, 492609224), (1378197108, 0.578758990150398, 2107752395), '0.09280790004748335') We are sending mail with results at report@fotonower.com args[1378196995] : ((1378196995, -0.5531050968037737, 492688767), (1378196995, 0.7913765723402356, 2107752395), '0.09280790004748335') We are sending mail with results at report@fotonower.com refus_total : 0.09280790004748335 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=26116300 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26116300_21-08-2025_10_41_51.pdf results_Auto_P26116300_21-08-2025_10_41_51.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26116300_21-08-2025_10_41_51.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','26116300','results_Auto_P26116300_21-08-2025_10_41_51.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26116300_21-08-2025_10_41_51.pdf','pdf','','0.38','0.09280790004748335') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/26116300

https://www.fotonower.com/image?json=false&list_photos_id=1378197880
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378197866
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378197144
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378197137
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378197128
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378197126
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378197125
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378197117
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378197115
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378197111
La photo est trop floue, merci de reprendre une photo.(avec le score = 3.7357849815636213)
https://www.fotonower.com/image?json=false&list_photos_id=1378197110
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378197108
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378196995
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 9.28%
Veuillez trouver les photos des contaminants.

exemples de contaminants: pet_clair: https://www.fotonower.com/view/26116501?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/26116502?limit=200
exemples de contaminants: autre: https://www.fotonower.com/view/26116503?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/26116507?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26116300_21-08-2025_10_41_51.pdf.

Lien vers velours :https://www.fotonower.com/velours/26116499,26116500,26116501,26116502,26116503,26116504,26116505,26116506,26116507,26116508,26116509?tags=environnement,pehd,pet_clair,papier,autre,pet_fonce,mal_croppe,metal,carton,background,flou.


L'équipe Fotonower 202 b'' Server: nginx Date: Thu, 21 Aug 2025 08:41:56 GMT Content-Length: 0 Connection: close X-Message-Id: AsBuBYW3SXODuHWVL59F3Q Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1378197880, 1378197866, 1378197144, 1378197137, 1378197128, 1378197126, 1378197125, 1378197117, 1378197115, 1378197111, 1378197110, 1378197108, 1378196995] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197880', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197866', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197144', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197137', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197128', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197126', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197125', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197117', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197115', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197111', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197110', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197108', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378196995', None, None, None, None, None, '3568032') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 13 time used for this insertion : 0.012958765029907227 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.756453990936279 time spend to save output : 0.01320028305053711 total time spend for step 9 : 4.769654273986816 step10:split_time_score Thu Aug 21 10:41:56 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('09', 2), ('10', 11)) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 21082025 26116300 Nombre de photos uploadées : 13 / 23040 (0%) 21082025 26116300 Nombre de photos taguées (types de déchets): 0 / 13 (0%) 21082025 26116300 Nombre de photos taguées (volume) : 0 / 13 (0%) elapsed_time : load_data_split_time_score 1.1920928955078125e-06 elapsed_time : order_list_meta_photo_and_scores 4.76837158203125e-06 ????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0006093978881835938 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.215224027633667 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.06594444444444442 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26113208_21-08-2025_08_21_34.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26113208 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26113208 AND mptpi.`type`=3594 To do Qualite : 0.14344428077066965 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26114421_21-08-2025_09_23_10.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26114421 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26114421 AND mptpi.`type`=3594 To do Qualite : 0.13331061385459533 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26114424_21-08-2025_09_11_26.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26114424 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26114424 AND mptpi.`type`=3594 To do Qualite : 0.06449963563100139 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26115111_21-08-2025_10_02_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26115111 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26115111 AND mptpi.`type`=3594 To do Qualite : 0.037599160420340956 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26115112_21-08-2025_10_12_16.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26115112 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26115112 AND mptpi.`type`=3594 To do Qualite : 0.05686577690972222 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26115114_21-08-2025_09_51_52.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26115114 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26115114 AND mptpi.`type`=3594 To do Qualite : 0.09280790004748335 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26116300_21-08-2025_10_41_51.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26116300 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26116300 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26116302 order by id desc limit 1 NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'21082025': {'nb_upload': 13, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1378197880, 1378197866, 1378197144, 1378197137, 1378197128, 1378197126, 1378197125, 1378197117, 1378197115, 1378197111, 1378197110, 1378197108, 1378196995] Looping around the photos to save general results len do output : 1 /26116300Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197880', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197866', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197144', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197137', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197128', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197126', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197125', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197117', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197115', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197111', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197110', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378197108', None, None, None, None, None, '3568032') ('3318', None, None, None, None, None, None, None, '3568032') ('3318', '26116300', '1378196995', None, None, None, None, None, '3568032') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 14 time used for this insertion : 0.01537942886352539 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.7175028324127197 time spend to save output : 0.01569223403930664 total time spend for step 10 : 1.7331950664520264 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 13 set_done_treatment 47.21user 23.66system 1:33.40elapsed 75%CPU (0avgtext+0avgdata 2842368maxresident)k 390024inputs+25232outputs (19major+1875352minor)pagefaults 0swaps