python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2689442 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['3762221'] with mtr_portfolio_ids : ['27120447'] and first list_photo_ids : [] new path : /proc/2689442/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 15 ; length of list_pids : 15 ; length of list_args : 15 time to download the photos : 2.8396008014678955 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Tue Sep 23 08:50:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10361 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-09-23 08:50:32.967743: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-09-23 08:50:32.996638: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-09-23 08:50:32.999426: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f1180000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-09-23 08:50:32.999521: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-09-23 08:50:33.005666: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-09-23 08:50:33.167814: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x169fe9a0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-09-23 08:50:33.167872: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-09-23 08:50:33.169725: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-23 08:50:33.170115: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-23 08:50:33.172984: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-23 08:50:33.176101: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-23 08:50:33.176480: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-23 08:50:33.180987: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-23 08:50:33.182728: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-23 08:50:33.189556: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-23 08:50:33.191514: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-23 08:50:33.191599: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-23 08:50:33.192584: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-23 08:50:33.192605: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-23 08:50:33.192616: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-23 08:50:33.194082: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9596 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-09-23 08:50:33.475967: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-23 08:50:33.476047: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-23 08:50:33.476068: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-23 08:50:33.476086: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-23 08:50:33.476104: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-23 08:50:33.476121: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-23 08:50:33.476139: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-23 08:50:33.476156: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-23 08:50:33.477715: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-23 08:50:33.479075: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-23 08:50:33.479110: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-23 08:50:33.479128: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-23 08:50:33.479145: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-23 08:50:33.479162: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-23 08:50:33.479179: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-23 08:50:33.479197: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-23 08:50:33.479214: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-23 08:50:33.480775: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-23 08:50:33.480808: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-23 08:50:33.480819: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-23 08:50:33.480828: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-23 08:50:33.482307: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9596 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-09-23 08:50:42.012463: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-23 08:50:42.222069: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 15 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 9.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 22.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 24.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 35.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 31.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 34.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 21.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 10.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 13 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 34.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 11 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 22.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 12 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 25.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 13 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 39.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 37.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 35.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 11 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 32.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 Detection mask done ! Trying to reset tf kernel 2689656 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 4031 tf kernel not reseted sub process len(results) : 15 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 15 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5480 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0002779960632324219 nb_pixel_total : 10245 time to create 1 rle with old method : 0.012315750122070312 length of segment : 108 time for calcul the mask position with numpy : 0.000301361083984375 nb_pixel_total : 12449 time to create 1 rle with old method : 0.015224695205688477 length of segment : 169 time for calcul the mask position with numpy : 0.0002713203430175781 nb_pixel_total : 12464 time to create 1 rle with old method : 0.014326333999633789 length of segment : 181 time for calcul the mask position with numpy : 0.00020074844360351562 nb_pixel_total : 4948 time to create 1 rle with old method : 0.0062067508697509766 length of segment : 91 time for calcul the mask position with numpy : 0.0002472400665283203 nb_pixel_total : 7331 time to create 1 rle with old method : 0.008919715881347656 length of segment : 94 time for calcul the mask position with numpy : 0.00022721290588378906 nb_pixel_total : 5180 time to create 1 rle with old method : 0.0065538883209228516 length of segment : 90 time for calcul the mask position with numpy : 0.0025794506072998047 nb_pixel_total : 104050 time to create 1 rle with old method : 0.11773514747619629 length of segment : 528 time for calcul the mask position with numpy : 0.0003936290740966797 nb_pixel_total : 11979 time to create 1 rle with old method : 0.014004707336425781 length of segment : 181 time for calcul the mask position with numpy : 0.00029754638671875 nb_pixel_total : 4060 time to create 1 rle with old method : 0.00820016860961914 length of segment : 96 time for calcul the mask position with numpy : 0.0006647109985351562 nb_pixel_total : 32091 time to create 1 rle with old method : 0.03459668159484863 length of segment : 270 time for calcul the mask position with numpy : 0.00040531158447265625 nb_pixel_total : 16849 time to create 1 rle with old method : 0.01804518699645996 length of segment : 200 time for calcul the mask position with numpy : 0.0009059906005859375 nb_pixel_total : 48385 time to create 1 rle with old method : 0.051885366439819336 length of segment : 288 time for calcul the mask position with numpy : 0.0007617473602294922 nb_pixel_total : 29037 time to create 1 rle with old method : 0.0321345329284668 length of segment : 222 time for calcul the mask position with numpy : 0.00041103363037109375 nb_pixel_total : 15378 time to create 1 rle with old method : 0.01692938804626465 length of segment : 198 time for calcul the mask position with numpy : 0.0003113746643066406 nb_pixel_total : 10836 time to create 1 rle with old method : 0.011884689331054688 length of segment : 171 time for calcul the mask position with numpy : 0.00022864341735839844 nb_pixel_total : 8846 time to create 1 rle with old method : 0.009791374206542969 length of segment : 135 time for calcul the mask position with numpy : 0.00030112266540527344 nb_pixel_total : 11451 time to create 1 rle with old method : 0.012350797653198242 length of segment : 176 time for calcul the mask position with numpy : 0.001767873764038086 nb_pixel_total : 98406 time to create 1 rle with old method : 0.10356998443603516 length of segment : 606 time for calcul the mask position with numpy : 0.0002613067626953125 nb_pixel_total : 8369 time to create 1 rle with old method : 0.009286642074584961 length of segment : 127 time for calcul the mask position with numpy : 0.0006749629974365234 nb_pixel_total : 33952 time to create 1 rle with old method : 0.037215471267700195 length of segment : 198 time for calcul the mask position with numpy : 0.002087116241455078 nb_pixel_total : 106310 time to create 1 rle with old method : 0.11446952819824219 length of segment : 534 time for calcul the mask position with numpy : 0.0001220703125 nb_pixel_total : 4298 time to create 1 rle with old method : 0.004862546920776367 length of segment : 45 time for calcul the mask position with numpy : 0.0003237724304199219 nb_pixel_total : 10729 time to create 1 rle with old method : 0.013660192489624023 length of segment : 87 time for calcul the mask position with numpy : 0.0006165504455566406 nb_pixel_total : 12712 time to create 1 rle with old method : 0.013471841812133789 length of segment : 185 time for calcul the mask position with numpy : 0.00019097328186035156 nb_pixel_total : 3294 time to create 1 rle with old method : 0.004424333572387695 length of segment : 64 time for calcul the mask position with numpy : 0.0002472400665283203 nb_pixel_total : 9077 time to create 1 rle with old method : 0.01086568832397461 length of segment : 135 time for calcul the mask position with numpy : 0.0010938644409179688 nb_pixel_total : 56373 time to create 1 rle with old method : 0.06446003913879395 length of segment : 223 time for calcul the mask position with numpy : 0.00043129920959472656 nb_pixel_total : 8847 time to create 1 rle with old method : 0.01185154914855957 length of segment : 157 time for calcul the mask position with numpy : 0.0003323554992675781 nb_pixel_total : 9025 time to create 1 rle with old method : 0.010034561157226562 length of segment : 91 time for calcul the mask position with numpy : 0.0024003982543945312 nb_pixel_total : 84770 time to create 1 rle with old method : 0.09026861190795898 length of segment : 474 time for calcul the mask position with numpy : 0.00025177001953125 nb_pixel_total : 4754 time to create 1 rle with old method : 0.005283832550048828 length of segment : 73 time for calcul the mask position with numpy : 0.00023746490478515625 nb_pixel_total : 4027 time to create 1 rle with old method : 0.004479885101318359 length of segment : 123 time for calcul the mask position with numpy : 0.0005865097045898438 nb_pixel_total : 13847 time to create 1 rle with old method : 0.015004873275756836 length of segment : 198 time for calcul the mask position with numpy : 0.0004100799560546875 nb_pixel_total : 11387 time to create 1 rle with old method : 0.012602567672729492 length of segment : 104 time for calcul the mask position with numpy : 0.013069629669189453 nb_pixel_total : 385599 time to create 1 rle with new method : 0.029584884643554688 length of segment : 1331 time for calcul the mask position with numpy : 0.0006799697875976562 nb_pixel_total : 25230 time to create 1 rle with old method : 0.02720499038696289 length of segment : 266 time for calcul the mask position with numpy : 0.0002486705780029297 nb_pixel_total : 6023 time to create 1 rle with old method : 0.006688356399536133 length of segment : 110 time for calcul the mask position with numpy : 0.00033783912658691406 nb_pixel_total : 7711 time to create 1 rle with old method : 0.008728742599487305 length of segment : 99 time for calcul the mask position with numpy : 0.00127410888671875 nb_pixel_total : 42024 time to create 1 rle with old method : 0.04487466812133789 length of segment : 322 time for calcul the mask position with numpy : 0.00021266937255859375 nb_pixel_total : 5714 time to create 1 rle with old method : 0.0064411163330078125 length of segment : 67 time for calcul the mask position with numpy : 0.0005486011505126953 nb_pixel_total : 12859 time to create 1 rle with old method : 0.014274358749389648 length of segment : 178 time for calcul the mask position with numpy : 8.559226989746094e-05 nb_pixel_total : 2329 time to create 1 rle with old method : 0.0026879310607910156 length of segment : 66 time for calcul the mask position with numpy : 0.0006444454193115234 nb_pixel_total : 13229 time to create 1 rle with old method : 0.01592850685119629 length of segment : 188 time for calcul the mask position with numpy : 0.00026798248291015625 nb_pixel_total : 6836 time to create 1 rle with old method : 0.007712125778198242 length of segment : 102 time for calcul the mask position with numpy : 0.000274658203125 nb_pixel_total : 7037 time to create 1 rle with old method : 0.007810115814208984 length of segment : 144 time spent for convertir_results : 2.531564712524414 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 45 chid ids of type : 3594 Number RLEs to save : 9495 save missing photos in datou_result : time spend for datou_step_exec : 27.35148286819458 time spend to save output : 0.6246964931488037 total time spend for step 1 : 27.976179361343384 step2:crop_condition Tue Sep 23 08:50:58 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 15 ! batch 1 Loaded 45 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 23 About to insert : list_path_to_insert length 23 new photo from crops ! About to upload 23 photos upload in portfolio : 3736932 init cache_photo without model_param we have 23 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758610260_2689442 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 23 photos in the portfolio 3736932 time of upload the photos Elapsed time : 16.465957403182983 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! About to upload 6 photos upload in portfolio : 3736932 init cache_photo without model_param we have 6 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758610279_2689442 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 6 photos in the portfolio 3736932 time of upload the photos Elapsed time : 2.2427408695220947 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3736932 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758610281_2689442 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 2 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.024970293045044 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 13 About to insert : list_path_to_insert length 13 new photo from crops ! About to upload 13 photos upload in portfolio : 3736932 init cache_photo without model_param we have 13 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758610285_2689442 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 13 photos in the portfolio 3736932 time of upload the photos Elapsed time : 4.260360240936279 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758610290_2689442 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.5163791179656982 we have finished the crop for the class : pet_fonce delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1385604706, 1385604703, 1385604700, 1385604679, 1385604662, 1385604642, 1385604636, 1385604630, 1385604618, 1385604416, 1385604415, 1385604411, 1385604387, 1385604385, 1385604381] Looping around the photos to save general results len do output : 45 /1385613664Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613665Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613666Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613667Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613669Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613671Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613672Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613674Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613675Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613676Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613678Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613679Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613680Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613682Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613683Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613684Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613685Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613687Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613688Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613689Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613691Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613692Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613693Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613721Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613722Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613723Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613725Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613726Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613727Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613736Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613738Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613789Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613790Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613792Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613793Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613795Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613796Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613798Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613800Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613801Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613803Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613805Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613806Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613808Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385613817Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604706', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604703', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604700', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604679', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604662', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604642', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604636', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604630', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604618', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604416', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604415', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604411', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604387', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604385', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604381', None, None, None, None, None, '3762221') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 150 time used for this insertion : 0.024323463439941406 save_final save missing photos in datou_result : time spend for datou_step_exec : 32.334163665771484 time spend to save output : 0.026370763778686523 total time spend for step 2 : 32.36053442955017 step3:rle_unique_nms_with_priority Tue Sep 23 08:51:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 45 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.40613436698913574 time for calcul the mask position with numpy : 0.0857844352722168 nb_pixel_total : 2050906 time to create 1 rle with new method : 0.09061980247497559 time for calcul the mask position with numpy : 0.006639242172241211 nb_pixel_total : 12449 time to create 1 rle with old method : 0.014070510864257812 time for calcul the mask position with numpy : 0.006644725799560547 nb_pixel_total : 10245 time to create 1 rle with old method : 0.011583805084228516 create new chi : 0.22620773315429688 time to delete rle : 0.020655393600463867 batch 1 Loaded 5 chid ids of type : 3594 +++Number RLEs to save : 1634 TO DO : save crop sub photo not yet done ! save time : 0.15093231201171875 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.05568337440490723 time for calcul the mask position with numpy : 0.10970425605773926 nb_pixel_total : 2061136 time to create 1 rle with new method : 0.23839235305786133 time for calcul the mask position with numpy : 0.009557723999023438 nb_pixel_total : 12464 time to create 1 rle with old method : 0.014221668243408203 create new chi : 0.3837466239929199 time to delete rle : 0.0003368854522705078 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 1442 TO DO : save crop sub photo not yet done ! save time : 0.11966395378112793 nb_obj : 6 nb_hashtags : 3 time to prepare the origin masks : 0.5763957500457764 time for calcul the mask position with numpy : 0.06394720077514648 nb_pixel_total : 1936052 time to create 1 rle with new method : 0.09740233421325684 time for calcul the mask position with numpy : 0.006300687789916992 nb_pixel_total : 4060 time to create 1 rle with old method : 0.004599332809448242 time for calcul the mask position with numpy : 0.0065038204193115234 nb_pixel_total : 11979 time to create 1 rle with old method : 0.01627826690673828 time for calcul the mask position with numpy : 0.008261680603027344 nb_pixel_total : 104050 time to create 1 rle with old method : 0.11868643760681152 time for calcul the mask position with numpy : 0.010632753372192383 nb_pixel_total : 5180 time to create 1 rle with old method : 0.00587916374206543 time for calcul the mask position with numpy : 0.010356426239013672 nb_pixel_total : 7331 time to create 1 rle with old method : 0.008272409439086914 time for calcul the mask position with numpy : 0.010598182678222656 nb_pixel_total : 4948 time to create 1 rle with old method : 0.00561976432800293 create new chi : 0.38486409187316895 time to delete rle : 0.0009005069732666016 batch 1 Loaded 13 chid ids of type : 3594 ++++++Number RLEs to save : 3240 TO DO : save crop sub photo not yet done ! save time : 0.23960661888122559 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.0651557445526123 time for calcul the mask position with numpy : 0.02060675621032715 nb_pixel_total : 2024660 time to create 1 rle with new method : 0.23532462120056152 time for calcul the mask position with numpy : 0.006232500076293945 nb_pixel_total : 16849 time to create 1 rle with old method : 0.019023418426513672 time for calcul the mask position with numpy : 0.006502389907836914 nb_pixel_total : 32091 time to create 1 rle with old method : 0.03589510917663574 create new chi : 0.32394909858703613 time to delete rle : 0.0003159046173095703 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 2020 TO DO : save crop sub photo not yet done ! save time : 0.16275310516357422 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.03675675392150879 time for calcul the mask position with numpy : 0.02151346206665039 nb_pixel_total : 2025215 time to create 1 rle with new method : 0.03113532066345215 time for calcul the mask position with numpy : 0.010981321334838867 nb_pixel_total : 48385 time to create 1 rle with old method : 0.059226274490356445 create new chi : 0.12317633628845215 time to delete rle : 0.0002944469451904297 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 1656 TO DO : save crop sub photo not yet done ! save time : 0.14415907859802246 nb_obj : 5 nb_hashtags : 2 time to prepare the origin masks : 0.49469733238220215 time for calcul the mask position with numpy : 0.20534396171569824 nb_pixel_total : 1998052 time to create 1 rle with new method : 0.265270471572876 time for calcul the mask position with numpy : 0.007981061935424805 nb_pixel_total : 11451 time to create 1 rle with old method : 0.012701034545898438 time for calcul the mask position with numpy : 0.006772041320800781 nb_pixel_total : 8846 time to create 1 rle with old method : 0.009975910186767578 time for calcul the mask position with numpy : 0.006880283355712891 nb_pixel_total : 10836 time to create 1 rle with old method : 0.012334585189819336 time for calcul the mask position with numpy : 0.006570100784301758 nb_pixel_total : 15378 time to create 1 rle with old method : 0.017153024673461914 time for calcul the mask position with numpy : 0.007467508316040039 nb_pixel_total : 29037 time to create 1 rle with old method : 0.03622794151306152 create new chi : 0.6063499450683594 time to delete rle : 0.0004954338073730469 batch 1 Loaded 11 chid ids of type : 3594 +++++Number RLEs to save : 2884 TO DO : save crop sub photo not yet done ! save time : 0.21552586555480957 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.0402979850769043 time for calcul the mask position with numpy : 0.019454479217529297 nb_pixel_total : 1975194 time to create 1 rle with new method : 0.10105013847351074 time for calcul the mask position with numpy : 0.007178306579589844 nb_pixel_total : 98406 time to create 1 rle with old method : 0.11078882217407227 create new chi : 0.23893046379089355 time to delete rle : 0.00041294097900390625 batch 1 Loaded 3 chid ids of type : 3594 +++Number RLEs to save : 2292 TO DO : save crop sub photo not yet done ! save time : 0.17218303680419922 nb_obj : 4 nb_hashtags : 3 time to prepare the origin masks : 0.42685699462890625 time for calcul the mask position with numpy : 0.26970815658569336 nb_pixel_total : 1920671 time to create 1 rle with new method : 0.326007604598999 time for calcul the mask position with numpy : 0.006273984909057617 nb_pixel_total : 4298 time to create 1 rle with old method : 0.00489044189453125 time for calcul the mask position with numpy : 0.007031440734863281 nb_pixel_total : 106310 time to create 1 rle with old method : 0.12263679504394531 time for calcul the mask position with numpy : 0.006825685501098633 nb_pixel_total : 33952 time to create 1 rle with old method : 0.038129568099975586 time for calcul the mask position with numpy : 0.006757497787475586 nb_pixel_total : 8369 time to create 1 rle with old method : 0.009915828704833984 create new chi : 0.8088424205780029 time to delete rle : 0.0005209445953369141 batch 1 Loaded 9 chid ids of type : 3594 ++++Number RLEs to save : 2888 TO DO : save crop sub photo not yet done ! save time : 0.21661591529846191 nb_obj : 6 nb_hashtags : 1 time to prepare the origin masks : 0.09036993980407715 time for calcul the mask position with numpy : 0.20596098899841309 nb_pixel_total : 1972825 time to create 1 rle with new method : 0.1119379997253418 time for calcul the mask position with numpy : 0.006375551223754883 nb_pixel_total : 8847 time to create 1 rle with old method : 0.009841203689575195 time for calcul the mask position with numpy : 0.006716489791870117 nb_pixel_total : 56116 time to create 1 rle with old method : 0.06245589256286621 time for calcul the mask position with numpy : 0.0063893795013427734 nb_pixel_total : 9077 time to create 1 rle with old method : 0.010175228118896484 time for calcul the mask position with numpy : 0.00726318359375 nb_pixel_total : 3294 time to create 1 rle with old method : 0.003835439682006836 time for calcul the mask position with numpy : 0.006325483322143555 nb_pixel_total : 12712 time to create 1 rle with old method : 0.014257431030273438 time for calcul the mask position with numpy : 0.006608247756958008 nb_pixel_total : 10729 time to create 1 rle with old method : 0.012441873550415039 create new chi : 0.48230433464050293 time to delete rle : 0.0003268718719482422 batch 1 Loaded 13 chid ids of type : 3594 ++++++++Number RLEs to save : 2728 TO DO : save crop sub photo not yet done ! save time : 0.1955280303955078 nb_obj : 4 nb_hashtags : 3 time to prepare the origin masks : 0.07928466796875 time for calcul the mask position with numpy : 0.1879103183746338 nb_pixel_total : 1971024 time to create 1 rle with new method : 0.15624737739562988 time for calcul the mask position with numpy : 0.006244659423828125 nb_pixel_total : 4027 time to create 1 rle with old method : 0.004369974136352539 time for calcul the mask position with numpy : 0.007385730743408203 nb_pixel_total : 4754 time to create 1 rle with old method : 0.005306243896484375 time for calcul the mask position with numpy : 0.007132291793823242 nb_pixel_total : 84770 time to create 1 rle with old method : 0.09152698516845703 time for calcul the mask position with numpy : 0.006886720657348633 nb_pixel_total : 9025 time to create 1 rle with old method : 0.009772300720214844 create new chi : 0.49414801597595215 time to delete rle : 0.0005600452423095703 batch 1 Loaded 9 chid ids of type : 3594 ++++Number RLEs to save : 2602 TO DO : save crop sub photo not yet done ! save time : 0.20107150077819824 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.11775493621826172 time for calcul the mask position with numpy : 0.31438636779785156 nb_pixel_total : 1662767 time to create 1 rle with new method : 0.09734320640563965 time for calcul the mask position with numpy : 0.008382558822631836 nb_pixel_total : 385599 time to create 1 rle with new method : 0.19644570350646973 time for calcul the mask position with numpy : 0.006312370300292969 nb_pixel_total : 11387 time to create 1 rle with old method : 0.012340784072875977 time for calcul the mask position with numpy : 0.006269216537475586 nb_pixel_total : 13847 time to create 1 rle with old method : 0.014997720718383789 create new chi : 0.6674301624298096 time to delete rle : 0.0005881786346435547 batch 1 Loaded 7 chid ids of type : 3594 ++++++Number RLEs to save : 4346 TO DO : save crop sub photo not yet done ! save time : 0.2895524501800537 No data in photo_id : 1385604411 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.05922698974609375 time for calcul the mask position with numpy : 0.08487176895141602 nb_pixel_total : 2042347 time to create 1 rle with new method : 0.11530613899230957 time for calcul the mask position with numpy : 0.006348848342895508 nb_pixel_total : 6023 time to create 1 rle with old method : 0.0069272518157958984 time for calcul the mask position with numpy : 0.006094217300415039 nb_pixel_total : 25230 time to create 1 rle with old method : 0.027983903884887695 create new chi : 0.25905919075012207 time to delete rle : 0.0002677440643310547 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 1832 TO DO : save crop sub photo not yet done ! save time : 0.14542198181152344 nb_obj : 5 nb_hashtags : 2 time to prepare the origin masks : 0.37333059310913086 time for calcul the mask position with numpy : 0.4001963138580322 nb_pixel_total : 2002963 time to create 1 rle with new method : 0.08865785598754883 time for calcul the mask position with numpy : 0.006498575210571289 nb_pixel_total : 2329 time to create 1 rle with old method : 0.00266265869140625 time for calcul the mask position with numpy : 0.006044149398803711 nb_pixel_total : 12859 time to create 1 rle with old method : 0.01427912712097168 time for calcul the mask position with numpy : 0.006106376647949219 nb_pixel_total : 5714 time to create 1 rle with old method : 0.006406307220458984 time for calcul the mask position with numpy : 0.00678253173828125 nb_pixel_total : 42024 time to create 1 rle with old method : 0.04535412788391113 time for calcul the mask position with numpy : 0.006048917770385742 nb_pixel_total : 7711 time to create 1 rle with old method : 0.008655548095703125 create new chi : 0.6089518070220947 time to delete rle : 0.0004267692565917969 batch 1 Loaded 11 chid ids of type : 3594 +++++Number RLEs to save : 2544 TO DO : save crop sub photo not yet done ! save time : 0.2014293670654297 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.053873300552368164 time for calcul the mask position with numpy : 0.022275686264038086 nb_pixel_total : 2046498 time to create 1 rle with new method : 0.08861136436462402 time for calcul the mask position with numpy : 0.006839275360107422 nb_pixel_total : 7037 time to create 1 rle with old method : 0.011314868927001953 time for calcul the mask position with numpy : 0.007602214813232422 nb_pixel_total : 6836 time to create 1 rle with old method : 0.011120319366455078 time for calcul the mask position with numpy : 0.007066965103149414 nb_pixel_total : 13229 time to create 1 rle with old method : 0.021996736526489258 create new chi : 0.1864030361175537 time to delete rle : 0.00035762786865234375 batch 1 Loaded 7 chid ids of type : 3594 +++Number RLEs to save : 1948 TO DO : save crop sub photo not yet done ! save time : 0.1399388313293457 map_output_result : {1385604706: (0.0, 'Should be the crop_list due to order', 0), 1385604703: (0.0, 'Should be the crop_list due to order', 0), 1385604700: (0.0, 'Should be the crop_list due to order', 0), 1385604679: (0.0, 'Should be the crop_list due to order', 0), 1385604662: (0.0, 'Should be the crop_list due to order', 0), 1385604642: (0.0, 'Should be the crop_list due to order', 0), 1385604636: (0.0, 'Should be the crop_list due to order', 0), 1385604630: (0.0, 'Should be the crop_list due to order', 0), 1385604618: (0.0, 'Should be the crop_list due to order', 0), 1385604416: (0.0, 'Should be the crop_list due to order', 0), 1385604415: (0.0, 'Should be the crop_list due to order', 0), 1385604411: (0.0, 'Should be the crop_list due to order', 0.0), 1385604387: (0.0, 'Should be the crop_list due to order', 0), 1385604385: (0.0, 'Should be the crop_list due to order', 0), 1385604381: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1385604706, 1385604703, 1385604700, 1385604679, 1385604662, 1385604642, 1385604636, 1385604630, 1385604618, 1385604416, 1385604415, 1385604411, 1385604387, 1385604385, 1385604381] Looping around the photos to save general results len do output : 15 /1385604706.Didn't retrieve data . /1385604703.Didn't retrieve data . /1385604700.Didn't retrieve data . /1385604679.Didn't retrieve data . /1385604662.Didn't retrieve data . /1385604642.Didn't retrieve data . /1385604636.Didn't retrieve data . /1385604630.Didn't retrieve data . /1385604618.Didn't retrieve data . /1385604416.Didn't retrieve data . /1385604415.Didn't retrieve data . /1385604411.Didn't retrieve data . /1385604387.Didn't retrieve data . /1385604385.Didn't retrieve data . /1385604381.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604706', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604703', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604700', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604679', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604662', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604642', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604636', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604630', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604618', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604416', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604415', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604411', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604387', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604385', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604381', None, None, None, None, None, '3762221') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 45 time used for this insertion : 0.016836881637573242 save_final save missing photos in datou_result : time spend for datou_step_exec : 11.7448570728302 time spend to save output : 0.01734757423400879 total time spend for step 3 : 11.762204647064209 step4:ventilate_hashtags_in_portfolio Tue Sep 23 08:51:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 27120447 get user id for portfolio 27120447 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27120447 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pehd','pet_fonce','papier','metal','flou','autre','carton','mal_croppe','environnement','pet_clair','background')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27120447 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pehd','pet_fonce','papier','metal','flou','autre','carton','mal_croppe','environnement','pet_clair','background')) AND mptpi.`min_score`=0.5 To do To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27120447 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pehd','pet_fonce','papier','metal','flou','autre','carton','mal_croppe','environnement','pet_clair','background')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://marlene.fotonower.com/velours/27121101,27121102,27121103,27121104,27121105,27121106,27121107,27121108,27121109,27121110,27121111?tags=pehd,pet_fonce,papier,metal,flou,autre,carton,mal_croppe,environnement,pet_clair,background Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1385604706, 1385604703, 1385604700, 1385604679, 1385604662, 1385604642, 1385604636, 1385604630, 1385604618, 1385604416, 1385604415, 1385604411, 1385604387, 1385604385, 1385604381] Looping around the photos to save general results len do output : 1 /27120447. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604706', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604703', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604700', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604679', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604662', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604642', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604636', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604630', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604618', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604416', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604415', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604411', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604387', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604385', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604381', None, None, None, None, None, '3762221') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 16 time used for this insertion : 0.017312288284301758 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.7527279853820801 time spend to save output : 0.017623424530029297 total time spend for step 4 : 0.7703514099121094 step5:final Tue Sep 23 08:51:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1385604706: ('0.04308416923868314',), 1385604703: ('0.04308416923868314',), 1385604700: ('0.04308416923868314',), 1385604679: ('0.04308416923868314',), 1385604662: ('0.04308416923868314',), 1385604642: ('0.04308416923868314',), 1385604636: ('0.04308416923868314',), 1385604630: ('0.04308416923868314',), 1385604618: ('0.04308416923868314',), 1385604416: ('0.04308416923868314',), 1385604415: ('0.04308416923868314',), 1385604411: ('0.04308416923868314',), 1385604387: ('0.04308416923868314',), 1385604385: ('0.04308416923868314',), 1385604381: ('0.04308416923868314',)} new output for save of step final : {1385604706: ('0.04308416923868314',), 1385604703: ('0.04308416923868314',), 1385604700: ('0.04308416923868314',), 1385604679: ('0.04308416923868314',), 1385604662: ('0.04308416923868314',), 1385604642: ('0.04308416923868314',), 1385604636: ('0.04308416923868314',), 1385604630: ('0.04308416923868314',), 1385604618: ('0.04308416923868314',), 1385604416: ('0.04308416923868314',), 1385604415: ('0.04308416923868314',), 1385604411: ('0.04308416923868314',), 1385604387: ('0.04308416923868314',), 1385604385: ('0.04308416923868314',), 1385604381: ('0.04308416923868314',)} [1385604706, 1385604703, 1385604700, 1385604679, 1385604662, 1385604642, 1385604636, 1385604630, 1385604618, 1385604416, 1385604415, 1385604411, 1385604387, 1385604385, 1385604381] Looping around the photos to save general results len do output : 15 /1385604706.Didn't retrieve data . /1385604703.Didn't retrieve data . /1385604700.Didn't retrieve data . /1385604679.Didn't retrieve data . /1385604662.Didn't retrieve data . /1385604642.Didn't retrieve data . /1385604636.Didn't retrieve data . /1385604630.Didn't retrieve data . /1385604618.Didn't retrieve data . /1385604416.Didn't retrieve data . /1385604415.Didn't retrieve data . /1385604411.Didn't retrieve data . /1385604387.Didn't retrieve data . /1385604385.Didn't retrieve data . /1385604381.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604706', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604703', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604700', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604679', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604662', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604642', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604636', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604630', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604618', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604416', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604415', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604411', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604387', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604385', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604381', None, None, None, None, None, '3762221') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 45 time used for this insertion : 0.013901710510253906 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.1226186752319336 time spend to save output : 0.014681339263916016 total time spend for step 5 : 0.1373000144958496 step6:blur_detection Tue Sep 23 08:51:43 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1758610227_2689442_1385604706_1da91fdf7e2248b7a2e0e59e0bc836bb.jpg resize: (1080, 1920) 1385604706 -3.894523545729592 treat image : temp/1758610227_2689442_1385604703_444fdf17615b33a0f5576ab563976739.jpg resize: (1080, 1920) 1385604703 -1.1515381383222971 treat image : temp/1758610227_2689442_1385604700_657696f98239d6b1737e032f9a504e47.jpg resize: (1080, 1920) 1385604700 2.8100939128419964 treat image : temp/1758610227_2689442_1385604679_d6a065c0981d09dd4b460fe378a45b87.jpg resize: (1080, 1920) 1385604679 -0.8093764488462282 treat image : temp/1758610227_2689442_1385604662_52ac4b0d07dea73a5c0fe643d05d937d.jpg resize: (1080, 1920) 1385604662 -1.2111475043380375 treat image : temp/1758610227_2689442_1385604642_f9d9ed9b1386816dc51cce34f016f867.jpg resize: (1080, 1920) 1385604642 -1.3056841780015924 treat image : temp/1758610227_2689442_1385604636_56a039cde2de6545e1775f96f53baa62.jpg resize: (1080, 1920) 1385604636 -0.5847031367678504 treat image : temp/1758610227_2689442_1385604630_ddb81b4dd00c4a72a58d9550ed927ed7.jpg resize: (1080, 1920) 1385604630 -4.318424912194042 treat image : temp/1758610227_2689442_1385604618_1cb75ea69be853633e4d77d0f4f09c32.jpg resize: (1080, 1920) 1385604618 -0.6210972328716807 treat image : temp/1758610227_2689442_1385604416_79f5673795ebda51ccf6504e71bb9d1f.jpg resize: (1080, 1920) 1385604416 -0.21410326135777852 treat image : temp/1758610227_2689442_1385604415_27bdea30f9450073810402d231bd6888.jpg resize: (1080, 1920) 1385604415 -1.4029657985856385 treat image : temp/1758610227_2689442_1385604411_b557fa34f9e0bb345430eeeffef28389.jpg resize: (1080, 1920) 1385604411 -1.0756460842598512 treat image : temp/1758610227_2689442_1385604387_c7bea246ee5b7cd65b9499db9deab87b.jpg resize: (1080, 1920) 1385604387 2.0329854953389828 treat image : temp/1758610227_2689442_1385604385_23761c71ad07815c668d63feacb68ffe.jpg resize: (1080, 1920) 1385604385 -0.5579549074174065 treat image : temp/1758610227_2689442_1385604381_8b2cb81e7feeb0ac65d24a78a596cdb8.jpg resize: (1080, 1920) 1385604381 -0.4413876066183889 treat image : temp/1758610227_2689442_1385604706_1da91fdf7e2248b7a2e0e59e0bc836bb_rle_crop_3971187688_0.png resize: (158, 122) 1385613664 -2.547719962831253 treat image : temp/1758610227_2689442_1385604703_444fdf17615b33a0f5576ab563976739_rle_crop_3971187689_0.png resize: (181, 111) 1385613665 -0.44081930542735476 treat image : temp/1758610227_2689442_1385604700_657696f98239d6b1737e032f9a504e47_rle_crop_3971187694_0.png resize: (181, 116) 1385613666 -0.5821374639855884 treat image : temp/1758610227_2689442_1385604700_657696f98239d6b1737e032f9a504e47_rle_crop_3971187692_0.png resize: (90, 101) 1385613667 -1.915910252665532 treat image : temp/1758610227_2689442_1385604700_657696f98239d6b1737e032f9a504e47_rle_crop_3971187690_0.png resize: (77, 103) 1385613669 -2.019607460867211 treat image : temp/1758610227_2689442_1385604700_657696f98239d6b1737e032f9a504e47_rle_crop_3971187695_0.png resize: (91, 89) 1385613671 -3.912309996207338 treat image : temp/1758610227_2689442_1385604642_f9d9ed9b1386816dc51cce34f016f867_rle_crop_3971187703_0.png resize: (176, 103) 1385613672 -0.4487944170587709 treat image : temp/1758610227_2689442_1385604642_f9d9ed9b1386816dc51cce34f016f867_rle_crop_3971187701_0.png resize: (169, 94) 1385613674 -1.6806932969012724 treat image : temp/1758610227_2689442_1385604642_f9d9ed9b1386816dc51cce34f016f867_rle_crop_3971187700_0.png resize: (190, 142) 1385613675 -1.4683491172882346 treat image : temp/1758610227_2689442_1385604630_ddb81b4dd00c4a72a58d9550ed927ed7_rle_crop_3971187705_0.png resize: (115, 142) 1385613676 -3.7225135473682283 treat image : temp/1758610227_2689442_1385604630_ddb81b4dd00c4a72a58d9550ed927ed7_rle_crop_3971187706_0.png resize: (196, 248) 1385613678 -5.190276996369072 treat image : temp/1758610227_2689442_1385604618_1cb75ea69be853633e4d77d0f4f09c32_rle_crop_3971187710_0.png resize: (185, 127) 1385613679 -0.574216299109459 treat image : temp/1758610227_2689442_1385604618_1cb75ea69be853633e4d77d0f4f09c32_rle_crop_3971187711_0.png resize: (56, 95) 1385613680 -3.063426768818156 treat image : temp/1758610227_2689442_1385604618_1cb75ea69be853633e4d77d0f4f09c32_rle_crop_3971187709_0.png resize: (85, 178) 1385613682 0.8420463087561307 treat image : temp/1758610227_2689442_1385604618_1cb75ea69be853633e4d77d0f4f09c32_rle_crop_3971187712_0.png resize: (116, 98) 1385613683 -2.224919269284677 treat image : temp/1758610227_2689442_1385604618_1cb75ea69be853633e4d77d0f4f09c32_rle_crop_3971187713_0.png resize: (195, 417) 1385613684 -2.72030184589738 treat image : temp/1758610227_2689442_1385604618_1cb75ea69be853633e4d77d0f4f09c32_rle_crop_3971187714_0.png resize: (157, 91) 1385613685 -1.9305380247243267 treat image : temp/1758610227_2689442_1385604416_79f5673795ebda51ccf6504e71bb9d1f_rle_crop_3971187715_0.png resize: (91, 128) 1385613687 0.8637097617509154 treat image : temp/1758610227_2689442_1385604415_27bdea30f9450073810402d231bd6888_rle_crop_3971187719_0.png resize: (184, 124) 1385613688 -2.259182205652793 treat image : temp/1758610227_2689442_1385604385_23761c71ad07815c668d63feacb68ffe_rle_crop_3971187727_0.png resize: (178, 130) 1385613689 -0.4775668543560264 treat image : temp/1758610227_2689442_1385604385_23761c71ad07815c668d63feacb68ffe_rle_crop_3971187724_0.png resize: (98, 115) 1385613691 -0.3245789044921919 treat image : temp/1758610227_2689442_1385604385_23761c71ad07815c668d63feacb68ffe_rle_crop_3971187728_0.png resize: (66, 53) 1385613692 0.36889282222178904 treat image : temp/1758610227_2689442_1385604381_8b2cb81e7feeb0ac65d24a78a596cdb8_rle_crop_3971187729_0.png resize: (188, 128) 1385613693 -0.4605126440164206 treat image : temp/1758610227_2689442_1385604706_1da91fdf7e2248b7a2e0e59e0bc836bb_rle_crop_3971187687_0.png resize: (106, 120) 1385613721 -2.2537427575310187 treat image : temp/1758610227_2689442_1385604636_56a039cde2de6545e1775f96f53baa62_rle_crop_3971187704_0.png resize: (536, 321) 1385613722 -0.5271195194301098 treat image : temp/1758610227_2689442_1385604416_79f5673795ebda51ccf6504e71bb9d1f_rle_crop_3971187718_0.png resize: (123, 47) 1385613723 -0.8163163337650016 treat image : temp/1758610227_2689442_1385604415_27bdea30f9450073810402d231bd6888_rle_crop_3971187721_0.png resize: (867, 939) 1385613725 -0.2693426178546742 treat image : temp/1758610227_2689442_1385604415_27bdea30f9450073810402d231bd6888_rle_crop_3971187720_0.png resize: (104, 138) 1385613726 0.2533912034936586 treat image : temp/1758610227_2689442_1385604387_c7bea246ee5b7cd65b9499db9deab87b_rle_crop_3971187723_0.png resize: (110, 81) 1385613727 -0.5771477817454919 treat image : temp/1758610227_2689442_1385604700_657696f98239d6b1737e032f9a504e47_rle_crop_3971187691_0.png resize: (94, 106) 1385613736 -1.6363368223446824 treat image : temp/1758610227_2689442_1385604630_ddb81b4dd00c4a72a58d9550ed927ed7_rle_crop_3971187708_0.png resize: (42, 138) 1385613738 -2.17007854830283 treat image : temp/1758610227_2689442_1385604700_657696f98239d6b1737e032f9a504e47_rle_crop_3971187693_0.png resize: (525, 319) 1385613789 0.2974720116844407 treat image : temp/1758610227_2689442_1385604679_d6a065c0981d09dd4b460fe378a45b87_rle_crop_3971187696_0.png resize: (270, 184) 1385613790 -1.6167786702725788 treat image : temp/1758610227_2689442_1385604662_52ac4b0d07dea73a5c0fe643d05d937d_rle_crop_3971187698_0.png resize: (268, 255) 1385613792 -0.705294861775446 treat image : temp/1758610227_2689442_1385604642_f9d9ed9b1386816dc51cce34f016f867_rle_crop_3971187699_0.png resize: (220, 176) 1385613793 -0.3623406946170455 treat image : temp/1758610227_2689442_1385604642_f9d9ed9b1386816dc51cce34f016f867_rle_crop_3971187702_0.png resize: (135, 80) 1385613795 1.3507943979592891 treat image : temp/1758610227_2689442_1385604630_ddb81b4dd00c4a72a58d9550ed927ed7_rle_crop_3971187707_0.png resize: (533, 343) 1385613796 0.038846393531750345 treat image : temp/1758610227_2689442_1385604416_79f5673795ebda51ccf6504e71bb9d1f_rle_crop_3971187717_0.png resize: (73, 87) 1385613798 -0.6624528028212807 treat image : temp/1758610227_2689442_1385604416_79f5673795ebda51ccf6504e71bb9d1f_rle_crop_3971187716_0.png resize: (466, 291) 1385613800 0.4781647984002257 treat image : temp/1758610227_2689442_1385604387_c7bea246ee5b7cd65b9499db9deab87b_rle_crop_3971187722_0.png resize: (265, 132) 1385613801 -1.0446864365669402 treat image : temp/1758610227_2689442_1385604385_23761c71ad07815c668d63feacb68ffe_rle_crop_3971187726_0.png resize: (66, 104) 1385613803 1.4887894328606908 treat image : temp/1758610227_2689442_1385604385_23761c71ad07815c668d63feacb68ffe_rle_crop_3971187725_0.png resize: (322, 199) 1385613805 -1.160590044445223 treat image : temp/1758610227_2689442_1385604381_8b2cb81e7feeb0ac65d24a78a596cdb8_rle_crop_3971187730_0.png resize: (93, 105) 1385613806 -0.26122964784973285 treat image : temp/1758610227_2689442_1385604381_8b2cb81e7feeb0ac65d24a78a596cdb8_rle_crop_3971187731_0.png resize: (138, 92) 1385613808 -1.9904844826288723 treat image : temp/1758610227_2689442_1385604679_d6a065c0981d09dd4b460fe378a45b87_rle_crop_3971187697_0.png resize: (192, 127) 1385613817 -1.731317246722153 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 60 time used for this insertion : 0.016018390655517578 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 60 time used for this insertion : 0.01758861541748047 save missing photos in datou_result : time spend for datou_step_exec : 12.804157257080078 time spend to save output : 0.03892016410827637 total time spend for step 6 : 12.843077421188354 step7:brightness Tue Sep 23 08:51:55 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1758610227_2689442_1385604706_1da91fdf7e2248b7a2e0e59e0bc836bb.jpg treat image : temp/1758610227_2689442_1385604703_444fdf17615b33a0f5576ab563976739.jpg treat image : temp/1758610227_2689442_1385604700_657696f98239d6b1737e032f9a504e47.jpg treat image : temp/1758610227_2689442_1385604679_d6a065c0981d09dd4b460fe378a45b87.jpg treat image : temp/1758610227_2689442_1385604662_52ac4b0d07dea73a5c0fe643d05d937d.jpg treat image : temp/1758610227_2689442_1385604642_f9d9ed9b1386816dc51cce34f016f867.jpg treat image : temp/1758610227_2689442_1385604636_56a039cde2de6545e1775f96f53baa62.jpg treat image : temp/1758610227_2689442_1385604630_ddb81b4dd00c4a72a58d9550ed927ed7.jpg treat image : temp/1758610227_2689442_1385604618_1cb75ea69be853633e4d77d0f4f09c32.jpg treat image : temp/1758610227_2689442_1385604416_79f5673795ebda51ccf6504e71bb9d1f.jpg treat image : temp/1758610227_2689442_1385604415_27bdea30f9450073810402d231bd6888.jpg treat image : temp/1758610227_2689442_1385604411_b557fa34f9e0bb345430eeeffef28389.jpg treat image : temp/1758610227_2689442_1385604387_c7bea246ee5b7cd65b9499db9deab87b.jpg treat image : temp/1758610227_2689442_1385604385_23761c71ad07815c668d63feacb68ffe.jpg treat image : temp/1758610227_2689442_1385604381_8b2cb81e7feeb0ac65d24a78a596cdb8.jpg treat image : temp/1758610227_2689442_1385604706_1da91fdf7e2248b7a2e0e59e0bc836bb_rle_crop_3971187688_0.png treat image : temp/1758610227_2689442_1385604703_444fdf17615b33a0f5576ab563976739_rle_crop_3971187689_0.png treat image : temp/1758610227_2689442_1385604700_657696f98239d6b1737e032f9a504e47_rle_crop_3971187694_0.png treat image : temp/1758610227_2689442_1385604700_657696f98239d6b1737e032f9a504e47_rle_crop_3971187692_0.png treat image : temp/1758610227_2689442_1385604700_657696f98239d6b1737e032f9a504e47_rle_crop_3971187690_0.png treat image : temp/1758610227_2689442_1385604700_657696f98239d6b1737e032f9a504e47_rle_crop_3971187695_0.png treat image : temp/1758610227_2689442_1385604642_f9d9ed9b1386816dc51cce34f016f867_rle_crop_3971187703_0.png treat image : temp/1758610227_2689442_1385604642_f9d9ed9b1386816dc51cce34f016f867_rle_crop_3971187701_0.png treat image : temp/1758610227_2689442_1385604642_f9d9ed9b1386816dc51cce34f016f867_rle_crop_3971187700_0.png treat image : temp/1758610227_2689442_1385604630_ddb81b4dd00c4a72a58d9550ed927ed7_rle_crop_3971187705_0.png treat image : temp/1758610227_2689442_1385604630_ddb81b4dd00c4a72a58d9550ed927ed7_rle_crop_3971187706_0.png treat image : temp/1758610227_2689442_1385604618_1cb75ea69be853633e4d77d0f4f09c32_rle_crop_3971187710_0.png treat image : temp/1758610227_2689442_1385604618_1cb75ea69be853633e4d77d0f4f09c32_rle_crop_3971187711_0.png treat image : temp/1758610227_2689442_1385604618_1cb75ea69be853633e4d77d0f4f09c32_rle_crop_3971187709_0.png treat image : temp/1758610227_2689442_1385604618_1cb75ea69be853633e4d77d0f4f09c32_rle_crop_3971187712_0.png treat image : temp/1758610227_2689442_1385604618_1cb75ea69be853633e4d77d0f4f09c32_rle_crop_3971187713_0.png treat image : temp/1758610227_2689442_1385604618_1cb75ea69be853633e4d77d0f4f09c32_rle_crop_3971187714_0.png treat image : temp/1758610227_2689442_1385604416_79f5673795ebda51ccf6504e71bb9d1f_rle_crop_3971187715_0.png treat image : temp/1758610227_2689442_1385604415_27bdea30f9450073810402d231bd6888_rle_crop_3971187719_0.png treat image : temp/1758610227_2689442_1385604385_23761c71ad07815c668d63feacb68ffe_rle_crop_3971187727_0.png treat image : temp/1758610227_2689442_1385604385_23761c71ad07815c668d63feacb68ffe_rle_crop_3971187724_0.png treat image : temp/1758610227_2689442_1385604385_23761c71ad07815c668d63feacb68ffe_rle_crop_3971187728_0.png treat image : temp/1758610227_2689442_1385604381_8b2cb81e7feeb0ac65d24a78a596cdb8_rle_crop_3971187729_0.png treat image : temp/1758610227_2689442_1385604706_1da91fdf7e2248b7a2e0e59e0bc836bb_rle_crop_3971187687_0.png treat image : temp/1758610227_2689442_1385604636_56a039cde2de6545e1775f96f53baa62_rle_crop_3971187704_0.png treat image : temp/1758610227_2689442_1385604416_79f5673795ebda51ccf6504e71bb9d1f_rle_crop_3971187718_0.png treat image : temp/1758610227_2689442_1385604415_27bdea30f9450073810402d231bd6888_rle_crop_3971187721_0.png treat image : temp/1758610227_2689442_1385604415_27bdea30f9450073810402d231bd6888_rle_crop_3971187720_0.png treat image : temp/1758610227_2689442_1385604387_c7bea246ee5b7cd65b9499db9deab87b_rle_crop_3971187723_0.png treat image : temp/1758610227_2689442_1385604700_657696f98239d6b1737e032f9a504e47_rle_crop_3971187691_0.png treat image : temp/1758610227_2689442_1385604630_ddb81b4dd00c4a72a58d9550ed927ed7_rle_crop_3971187708_0.png treat image : temp/1758610227_2689442_1385604700_657696f98239d6b1737e032f9a504e47_rle_crop_3971187693_0.png treat image : temp/1758610227_2689442_1385604679_d6a065c0981d09dd4b460fe378a45b87_rle_crop_3971187696_0.png treat image : temp/1758610227_2689442_1385604662_52ac4b0d07dea73a5c0fe643d05d937d_rle_crop_3971187698_0.png treat image : temp/1758610227_2689442_1385604642_f9d9ed9b1386816dc51cce34f016f867_rle_crop_3971187699_0.png treat image : temp/1758610227_2689442_1385604642_f9d9ed9b1386816dc51cce34f016f867_rle_crop_3971187702_0.png treat image : temp/1758610227_2689442_1385604630_ddb81b4dd00c4a72a58d9550ed927ed7_rle_crop_3971187707_0.png treat image : temp/1758610227_2689442_1385604416_79f5673795ebda51ccf6504e71bb9d1f_rle_crop_3971187717_0.png treat image : temp/1758610227_2689442_1385604416_79f5673795ebda51ccf6504e71bb9d1f_rle_crop_3971187716_0.png treat image : temp/1758610227_2689442_1385604387_c7bea246ee5b7cd65b9499db9deab87b_rle_crop_3971187722_0.png treat image : temp/1758610227_2689442_1385604385_23761c71ad07815c668d63feacb68ffe_rle_crop_3971187726_0.png treat image : temp/1758610227_2689442_1385604385_23761c71ad07815c668d63feacb68ffe_rle_crop_3971187725_0.png treat image : temp/1758610227_2689442_1385604381_8b2cb81e7feeb0ac65d24a78a596cdb8_rle_crop_3971187730_0.png treat image : temp/1758610227_2689442_1385604381_8b2cb81e7feeb0ac65d24a78a596cdb8_rle_crop_3971187731_0.png treat image : temp/1758610227_2689442_1385604679_d6a065c0981d09dd4b460fe378a45b87_rle_crop_3971187697_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 60 time used for this insertion : 0.015261173248291016 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 60 time used for this insertion : 0.01675271987915039 save missing photos in datou_result : time spend for datou_step_exec : 3.6746528148651123 time spend to save output : 0.036586761474609375 total time spend for step 7 : 3.7112395763397217 step8:velours_tree Tue Sep 23 08:51:59 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.13196325302124023 time spend to save output : 5.245208740234375e-05 total time spend for step 8 : 0.13201570510864258 step9:send_mail_cod Tue Sep 23 08:51:59 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P27120447_23-09-2025_08_51_59.pdf 27121101 imagette271211011758610319 27121102 change filename to text .imagette271211021758610319 27121103 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette271211031758610319 27121104 change filename to text .change filename to text .imagette271211041758610321 27121105 imagette271211051758610321 27121106 imagette271211061758610321 27121107 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette271211071758610321 27121108 imagette271211081758610321 27121110 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette271211101758610321 27121111 imagette271211111758610322 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=27120447 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://marlene.fotonower.com/velours/27121101,27121102,27121103,27121104,27121105,27121106,27121107,27121108,27121109,27121110,27121111?tags=pehd,pet_fonce,papier,metal,flou,autre,carton,mal_croppe,environnement,pet_clair,background args[1385604706] : ((1385604706, -3.894523545729592, 492609224), (1385604706, 0.5020941065493512, 2107752395), '0.04308416923868314') We are sending mail with results at report@fotonower.com args[1385604703] : ((1385604703, -1.1515381383222971, 492688767), (1385604703, 0.5354662730772234, 2107752395), '0.04308416923868314') We are sending mail with results at report@fotonower.com args[1385604700] : ((1385604700, 2.8100939128419964, 492688767), (1385604700, 0.22814520740816616, 2107752395), '0.04308416923868314') We are sending mail with results at report@fotonower.com args[1385604679] : ((1385604679, -0.8093764488462282, 492688767), (1385604679, 0.7848924075317177, 2107752395), '0.04308416923868314') We are sending mail with results at report@fotonower.com args[1385604662] : ((1385604662, -1.2111475043380375, 492688767), (1385604662, 0.7785521014901162, 2107752395), '0.04308416923868314') We are sending mail with results at report@fotonower.com args[1385604642] : ((1385604642, -1.3056841780015924, 492688767), (1385604642, 0.5176819026107767, 2107752395), '0.04308416923868314') We are sending mail with results at report@fotonower.com args[1385604636] : ((1385604636, -0.5847031367678504, 492688767), (1385604636, 0.3593233868494421, 2107752395), '0.04308416923868314') We are sending mail with results at report@fotonower.com args[1385604630] : ((1385604630, -4.318424912194042, 492609224), (1385604630, 0.3287301178032218, 2107752395), '0.04308416923868314') We are sending mail with results at report@fotonower.com args[1385604618] : ((1385604618, -0.6210972328716807, 492688767), (1385604618, 0.6952313437449531, 2107752395), '0.04308416923868314') We are sending mail with results at report@fotonower.com args[1385604416] : ((1385604416, -0.21410326135777852, 492688767), (1385604416, 0.7272309800454876, 2107752395), '0.04308416923868314') We are sending mail with results at report@fotonower.com args[1385604415] : ((1385604415, -1.4029657985856385, 492688767), (1385604415, 0.8240199701251832, 2107752395), '0.04308416923868314') We are sending mail with results at report@fotonower.com args[1385604411] : ((1385604411, -1.0756460842598512, 492688767), (1385604411, 0.7877113409638417, 2107752395), '0.04308416923868314') We are sending mail with results at report@fotonower.com args[1385604387] : ((1385604387, 2.0329854953389828, 492688767), (1385604387, 0.7210138643298077, 2107752395), '0.04308416923868314') We are sending mail with results at report@fotonower.com args[1385604385] : ((1385604385, -0.5579549074174065, 492688767), (1385604385, 0.6157530038934631, 2107752395), '0.04308416923868314') We are sending mail with results at report@fotonower.com args[1385604381] : ((1385604381, -0.4413876066183889, 492688767), (1385604381, 0.6113955532178498, 2107752395), '0.04308416923868314') We are sending mail with results at report@fotonower.com refus_total : 0.04308416923868314 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=27120447 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27120447_23-09-2025_08_51_59.pdf results_Auto_P27120447_23-09-2025_08_51_59.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27120447_23-09-2025_08_51_59.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','27120447','results_Auto_P27120447_23-09-2025_08_51_59.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27120447_23-09-2025_08_51_59.pdf','pdf','','0.29','0.04308416923868314') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/27120447

https://www.fotonower.com/image?json=false&list_photos_id=1385604706
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1385604703
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1385604700
La photo est trop floue, merci de reprendre une photo.(avec le score = 2.8100939128419964)
https://www.fotonower.com/image?json=false&list_photos_id=1385604679
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1385604662
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1385604642
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1385604636
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1385604630
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1385604618
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1385604416
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1385604415
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1385604411
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1385604387
La photo est trop floue, merci de reprendre une photo.(avec le score = 2.0329854953389828)
https://www.fotonower.com/image?json=false&list_photos_id=1385604385
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1385604381
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 4.31%
Veuillez trouver les photos des contaminants.

exemples de contaminants: pet_fonce: https://www.fotonower.com/view/27121102?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/27121103?limit=200
exemples de contaminants: metal: https://www.fotonower.com/view/27121104?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/27121107?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/27121110?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27120447_23-09-2025_08_51_59.pdf.

Lien vers velours :https://marlene.fotonower.com/velours/27121101,27121102,27121103,27121104,27121105,27121106,27121107,27121108,27121109,27121110,27121111?tags=pehd,pet_fonce,papier,metal,flou,autre,carton,mal_croppe,environnement,pet_clair,background.


L'équipe Fotonower 202 b'' Server: nginx Date: Tue, 23 Sep 2025 06:52:04 GMT Content-Length: 0 Connection: close X-Message-Id: umdFLXXeTZ21ZdJapC8MFw Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1385604706, 1385604703, 1385604700, 1385604679, 1385604662, 1385604642, 1385604636, 1385604630, 1385604618, 1385604416, 1385604415, 1385604411, 1385604387, 1385604385, 1385604381] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604706', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604703', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604700', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604679', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604662', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604642', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604636', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604630', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604618', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604416', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604415', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604411', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604387', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604385', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604381', None, None, None, None, None, '3762221') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 15 time used for this insertion : 0.016588926315307617 save_final save missing photos in datou_result : time spend for datou_step_exec : 5.003426790237427 time spend to save output : 0.01686239242553711 total time spend for step 9 : 5.020289182662964 step10:split_time_score Tue Sep 23 08:52:04 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('07', 15),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 23092025 27120447 Nombre de photos uploadées : 15 / 23040 (0%) 23092025 27120447 Nombre de photos taguées (types de déchets): 0 / 15 (0%) 23092025 27120447 Nombre de photos taguées (volume) : 0 / 15 (0%) elapsed_time : load_data_split_time_score 1.9073486328125e-06 elapsed_time : order_list_meta_photo_and_scores 5.4836273193359375e-06 ??????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0004513263702392578 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.21796631813049316 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27120440 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27120443 order by id desc limit 1 Qualite : 0.04308416923868314 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27120447_23-09-2025_08_51_59.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27120447 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27120447 AND mptpi.`type`=3594 To do Qualite : 0.06695908224400872 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27120448_23-09-2025_08_42_17.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27120448 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27120448 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'23092025': {'nb_upload': 15, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1385604706, 1385604703, 1385604700, 1385604679, 1385604662, 1385604642, 1385604636, 1385604630, 1385604618, 1385604416, 1385604415, 1385604411, 1385604387, 1385604385, 1385604381] Looping around the photos to save general results len do output : 1 /27120447Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604706', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604703', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604700', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604679', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604662', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604642', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604636', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604630', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604618', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604416', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604415', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604411', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604387', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604385', None, None, None, None, None, '3762221') ('3318', None, None, None, None, None, None, None, '3762221') ('3318', '27120447', '1385604381', None, None, None, None, None, '3762221') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 16 time used for this insertion : 0.01657867431640625 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.5609922409057617 time spend to save output : 0.016872406005859375 total time spend for step 10 : 2.577864646911621 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 15 set_done_treatment 46.60user 23.27system 1:42.76elapsed 68%CPU (0avgtext+0avgdata 2952412maxresident)k 540792inputs+20552outputs (483major+1959553minor)pagefaults 0swaps