python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 1414272 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['3717515'] with mtr_portfolio_ids : ['26824390'] and first list_photo_ids : [] new path : /proc/1414272/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 7 ; length of list_pids : 7 ; length of list_args : 7 time to download the photos : 2.0524051189422607 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Sat Sep 13 08:50:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10592 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-09-13 08:50:32.295627: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-09-13 08:50:32.303326: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-09-13 08:50:32.304770: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f92e8000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-09-13 08:50:32.304792: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-09-13 08:50:32.307489: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-09-13 08:50:32.457947: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x3531d180 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-09-13 08:50:32.458004: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-09-13 08:50:32.459279: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-13 08:50:32.459734: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-13 08:50:32.463019: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-13 08:50:32.466116: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-13 08:50:32.466512: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-13 08:50:32.469238: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-13 08:50:32.470626: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-13 08:50:32.475643: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-13 08:50:32.477181: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-13 08:50:32.477256: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-13 08:50:32.478037: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-13 08:50:32.478054: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-13 08:50:32.478063: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-13 08:50:32.479378: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9811 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-09-13 08:50:32.767450: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-13 08:50:32.767550: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-13 08:50:32.767575: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-13 08:50:32.767597: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-13 08:50:32.767617: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-13 08:50:32.767638: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-13 08:50:32.767658: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-13 08:50:32.767678: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-13 08:50:32.769298: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-13 08:50:32.770726: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-13 08:50:32.770783: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-13 08:50:32.770802: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-13 08:50:32.770820: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-13 08:50:32.770837: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-13 08:50:32.770854: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-13 08:50:32.770871: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-13 08:50:32.770888: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-13 08:50:32.772480: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-13 08:50:32.772517: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-13 08:50:32.772528: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-13 08:50:32.772538: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-13 08:50:32.774172: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9811 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-09-13 08:50:44.723360: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-13 08:50:44.922907: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 7 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 21 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 31 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 27 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 25 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 20 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 24 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 23 Detection mask done ! Trying to reset tf kernel 1414869 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 331 tf kernel not reseted sub process len(results) : 7 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 7 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 6813 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0017666816711425781 nb_pixel_total : 64065 time to create 1 rle with old method : 0.07300162315368652 length of segment : 196 time for calcul the mask position with numpy : 0.0010676383972167969 nb_pixel_total : 55903 time to create 1 rle with old method : 0.061553239822387695 length of segment : 341 time for calcul the mask position with numpy : 0.0012278556823730469 nb_pixel_total : 66183 time to create 1 rle with old method : 0.07219171524047852 length of segment : 369 time for calcul the mask position with numpy : 0.002804279327392578 nb_pixel_total : 110066 time to create 1 rle with old method : 0.11654329299926758 length of segment : 411 time for calcul the mask position with numpy : 0.0006000995635986328 nb_pixel_total : 23447 time to create 1 rle with old method : 0.02526998519897461 length of segment : 220 time for calcul the mask position with numpy : 0.0015935897827148438 nb_pixel_total : 72036 time to create 1 rle with old method : 0.0781259536743164 length of segment : 649 time for calcul the mask position with numpy : 0.0004744529724121094 nb_pixel_total : 26572 time to create 1 rle with old method : 0.029409170150756836 length of segment : 248 time for calcul the mask position with numpy : 0.0018155574798583984 nb_pixel_total : 106724 time to create 1 rle with old method : 0.11782145500183105 length of segment : 689 time for calcul the mask position with numpy : 0.0009889602661132812 nb_pixel_total : 45307 time to create 1 rle with old method : 0.0488283634185791 length of segment : 317 time for calcul the mask position with numpy : 0.0006806850433349609 nb_pixel_total : 30352 time to create 1 rle with old method : 0.04393458366394043 length of segment : 238 time for calcul the mask position with numpy : 0.0011053085327148438 nb_pixel_total : 49127 time to create 1 rle with old method : 0.05618906021118164 length of segment : 263 time for calcul the mask position with numpy : 0.0018215179443359375 nb_pixel_total : 96028 time to create 1 rle with old method : 0.10375714302062988 length of segment : 454 time for calcul the mask position with numpy : 0.0008249282836914062 nb_pixel_total : 38700 time to create 1 rle with old method : 0.04356718063354492 length of segment : 274 time for calcul the mask position with numpy : 0.000308990478515625 nb_pixel_total : 14284 time to create 1 rle with old method : 0.016162872314453125 length of segment : 142 time for calcul the mask position with numpy : 0.0010280609130859375 nb_pixel_total : 50802 time to create 1 rle with old method : 0.060862064361572266 length of segment : 323 time for calcul the mask position with numpy : 0.004315853118896484 nb_pixel_total : 171779 time to create 1 rle with new method : 0.011032581329345703 length of segment : 603 time for calcul the mask position with numpy : 0.0005812644958496094 nb_pixel_total : 22757 time to create 1 rle with old method : 0.024900436401367188 length of segment : 145 time for calcul the mask position with numpy : 0.0006344318389892578 nb_pixel_total : 36853 time to create 1 rle with old method : 0.03970789909362793 length of segment : 207 time for calcul the mask position with numpy : 0.00047707557678222656 nb_pixel_total : 24621 time to create 1 rle with old method : 0.025967121124267578 length of segment : 189 time for calcul the mask position with numpy : 0.00021958351135253906 nb_pixel_total : 9999 time to create 1 rle with old method : 0.011747598648071289 length of segment : 91 time for calcul the mask position with numpy : 0.001596689224243164 nb_pixel_total : 78999 time to create 1 rle with old method : 0.08476066589355469 length of segment : 484 time for calcul the mask position with numpy : 0.00044035911560058594 nb_pixel_total : 19331 time to create 1 rle with old method : 0.021238327026367188 length of segment : 194 time for calcul the mask position with numpy : 0.0018444061279296875 nb_pixel_total : 90837 time to create 1 rle with old method : 0.09579348564147949 length of segment : 308 time for calcul the mask position with numpy : 0.0006794929504394531 nb_pixel_total : 29801 time to create 1 rle with old method : 0.0335087776184082 length of segment : 346 time for calcul the mask position with numpy : 0.0007195472717285156 nb_pixel_total : 22679 time to create 1 rle with old method : 0.032881736755371094 length of segment : 196 time for calcul the mask position with numpy : 0.0006279945373535156 nb_pixel_total : 21823 time to create 1 rle with old method : 0.030919790267944336 length of segment : 265 time for calcul the mask position with numpy : 0.0017278194427490234 nb_pixel_total : 77092 time to create 1 rle with old method : 0.10895466804504395 length of segment : 405 time for calcul the mask position with numpy : 0.0005650520324707031 nb_pixel_total : 22943 time to create 1 rle with old method : 0.029793500900268555 length of segment : 183 time for calcul the mask position with numpy : 0.001238107681274414 nb_pixel_total : 53036 time to create 1 rle with old method : 0.06120729446411133 length of segment : 326 time for calcul the mask position with numpy : 0.0015099048614501953 nb_pixel_total : 99644 time to create 1 rle with old method : 0.10960912704467773 length of segment : 364 time for calcul the mask position with numpy : 0.0004482269287109375 nb_pixel_total : 13961 time to create 1 rle with old method : 0.017139673233032227 length of segment : 150 time for calcul the mask position with numpy : 0.0022230148315429688 nb_pixel_total : 68364 time to create 1 rle with old method : 0.07847929000854492 length of segment : 377 time for calcul the mask position with numpy : 0.0013010501861572266 nb_pixel_total : 61818 time to create 1 rle with old method : 0.06716275215148926 length of segment : 439 time for calcul the mask position with numpy : 0.0022542476654052734 nb_pixel_total : 121710 time to create 1 rle with old method : 0.13088107109069824 length of segment : 408 time for calcul the mask position with numpy : 0.0005984306335449219 nb_pixel_total : 23483 time to create 1 rle with old method : 0.02821063995361328 length of segment : 180 time for calcul the mask position with numpy : 0.0015382766723632812 nb_pixel_total : 64473 time to create 1 rle with old method : 0.0742032527923584 length of segment : 302 time for calcul the mask position with numpy : 0.0015110969543457031 nb_pixel_total : 45191 time to create 1 rle with old method : 0.05101633071899414 length of segment : 229 time for calcul the mask position with numpy : 0.0036323070526123047 nb_pixel_total : 112238 time to create 1 rle with old method : 0.12663626670837402 length of segment : 321 time for calcul the mask position with numpy : 0.009305953979492188 nb_pixel_total : 193916 time to create 1 rle with new method : 0.019991636276245117 length of segment : 636 time for calcul the mask position with numpy : 0.002923727035522461 nb_pixel_total : 123637 time to create 1 rle with old method : 0.13949823379516602 length of segment : 520 time for calcul the mask position with numpy : 0.005529165267944336 nb_pixel_total : 189955 time to create 1 rle with new method : 0.015990734100341797 length of segment : 917 time for calcul the mask position with numpy : 0.0008382797241210938 nb_pixel_total : 10561 time to create 1 rle with old method : 0.011864185333251953 length of segment : 188 time for calcul the mask position with numpy : 0.003503084182739258 nb_pixel_total : 133757 time to create 1 rle with old method : 0.1429588794708252 length of segment : 363 time for calcul the mask position with numpy : 0.0046575069427490234 nb_pixel_total : 142841 time to create 1 rle with old method : 0.15218472480773926 length of segment : 466 time for calcul the mask position with numpy : 0.004266262054443359 nb_pixel_total : 127731 time to create 1 rle with old method : 0.13667583465576172 length of segment : 425 time for calcul the mask position with numpy : 0.0029516220092773438 nb_pixel_total : 75540 time to create 1 rle with old method : 0.08394050598144531 length of segment : 703 time for calcul the mask position with numpy : 0.0008003711700439453 nb_pixel_total : 39237 time to create 1 rle with old method : 0.04191136360168457 length of segment : 387 time for calcul the mask position with numpy : 0.0021708011627197266 nb_pixel_total : 125707 time to create 1 rle with old method : 0.13879132270812988 length of segment : 460 time for calcul the mask position with numpy : 0.0014693737030029297 nb_pixel_total : 37085 time to create 1 rle with old method : 0.041730403900146484 length of segment : 227 time for calcul the mask position with numpy : 0.0012905597686767578 nb_pixel_total : 80674 time to create 1 rle with old method : 0.08910536766052246 length of segment : 394 time for calcul the mask position with numpy : 0.0020105838775634766 nb_pixel_total : 105714 time to create 1 rle with old method : 0.1135551929473877 length of segment : 447 time for calcul the mask position with numpy : 0.0003180503845214844 nb_pixel_total : 16304 time to create 1 rle with old method : 0.018250703811645508 length of segment : 114 time for calcul the mask position with numpy : 0.0008873939514160156 nb_pixel_total : 50092 time to create 1 rle with old method : 0.05601859092712402 length of segment : 307 time for calcul the mask position with numpy : 0.0003006458282470703 nb_pixel_total : 13842 time to create 1 rle with old method : 0.015558481216430664 length of segment : 131 time for calcul the mask position with numpy : 0.0006210803985595703 nb_pixel_total : 43166 time to create 1 rle with old method : 0.05008077621459961 length of segment : 138 time for calcul the mask position with numpy : 0.0016248226165771484 nb_pixel_total : 83932 time to create 1 rle with old method : 0.09473752975463867 length of segment : 349 time for calcul the mask position with numpy : 0.0005905628204345703 nb_pixel_total : 29643 time to create 1 rle with old method : 0.03457283973693848 length of segment : 246 time for calcul the mask position with numpy : 0.0011906623840332031 nb_pixel_total : 42136 time to create 1 rle with old method : 0.04892420768737793 length of segment : 273 time for calcul the mask position with numpy : 0.0010080337524414062 nb_pixel_total : 59657 time to create 1 rle with old method : 0.07025766372680664 length of segment : 334 time for calcul the mask position with numpy : 0.0010302066802978516 nb_pixel_total : 17978 time to create 1 rle with old method : 0.02051258087158203 length of segment : 219 time for calcul the mask position with numpy : 0.0006992816925048828 nb_pixel_total : 17706 time to create 1 rle with old method : 0.019921541213989258 length of segment : 149 time for calcul the mask position with numpy : 0.0024967193603515625 nb_pixel_total : 95408 time to create 1 rle with old method : 0.10381412506103516 length of segment : 281 time for calcul the mask position with numpy : 0.0012238025665283203 nb_pixel_total : 40214 time to create 1 rle with old method : 0.04320168495178223 length of segment : 221 time for calcul the mask position with numpy : 0.0004248619079589844 nb_pixel_total : 13727 time to create 1 rle with old method : 0.015003204345703125 length of segment : 137 time for calcul the mask position with numpy : 0.0016641616821289062 nb_pixel_total : 52243 time to create 1 rle with old method : 0.05584549903869629 length of segment : 248 time for calcul the mask position with numpy : 0.0009756088256835938 nb_pixel_total : 31094 time to create 1 rle with old method : 0.033440351486206055 length of segment : 195 time for calcul the mask position with numpy : 0.0022127628326416016 nb_pixel_total : 67500 time to create 1 rle with old method : 0.0727839469909668 length of segment : 310 time for calcul the mask position with numpy : 0.003364086151123047 nb_pixel_total : 103553 time to create 1 rle with old method : 0.11265993118286133 length of segment : 280 time for calcul the mask position with numpy : 0.0028345584869384766 nb_pixel_total : 94963 time to create 1 rle with old method : 0.10823750495910645 length of segment : 356 time for calcul the mask position with numpy : 0.0011861324310302734 nb_pixel_total : 28902 time to create 1 rle with old method : 0.03238391876220703 length of segment : 365 time for calcul the mask position with numpy : 0.0011222362518310547 nb_pixel_total : 30174 time to create 1 rle with old method : 0.034836530685424805 length of segment : 172 time for calcul the mask position with numpy : 0.0005519390106201172 nb_pixel_total : 19069 time to create 1 rle with old method : 0.022167682647705078 length of segment : 187 time for calcul the mask position with numpy : 0.0006611347198486328 nb_pixel_total : 15019 time to create 1 rle with old method : 0.017548084259033203 length of segment : 145 time for calcul the mask position with numpy : 0.0025844573974609375 nb_pixel_total : 91317 time to create 1 rle with old method : 0.12038755416870117 length of segment : 419 time for calcul the mask position with numpy : 0.0019271373748779297 nb_pixel_total : 58806 time to create 1 rle with old method : 0.06377887725830078 length of segment : 305 time spent for convertir_results : 8.533054113388062 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 75 chid ids of type : 3594 Number RLEs to save : 23860 save missing photos in datou_result : time spend for datou_step_exec : 53.60673403739929 time spend to save output : 2.493773937225342 total time spend for step 1 : 56.100507974624634 step2:crop_condition Sat Sep 13 08:51:25 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 7 ! batch 1 Loaded 75 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 39 About to insert : list_path_to_insert length 39 new photo from crops ! About to upload 39 photos upload in portfolio : 3736932 init cache_photo without model_param we have 39 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1757746297_1414272 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 39 photos in the portfolio 3736932 time of upload the photos Elapsed time : 10.935805082321167 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! About to upload 8 photos upload in portfolio : 3736932 init cache_photo without model_param we have 8 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1757746311_1414272 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 8 photos in the portfolio 3736932 time of upload the photos Elapsed time : 2.2163753509521484 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3736932 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1757746314_1414272 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 2 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.7599365711212158 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 24 About to insert : list_path_to_insert length 24 new photo from crops ! About to upload 24 photos upload in portfolio : 3736932 init cache_photo without model_param we have 24 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1757746325_1414272 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 24 photos in the portfolio 3736932 time of upload the photos Elapsed time : 6.801068067550659 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3736932 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1757746333_1414272 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 2 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.7849447727203369 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1383295201, 1383295159, 1383295117, 1383295080, 1383295037, 1383294996, 1383294879] Looping around the photos to save general results len do output : 75 /1383616130Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616131Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616132Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616133Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616134Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616135Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616136Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616137Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616138Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616139Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616140Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616141Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616142Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616143Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616144Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616145Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616146Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616147Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616148Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616149Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616151Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616152Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616153Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616154Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616156Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616157Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616158Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616159Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616160Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616161Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616162Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616163Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616164Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616165Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616166Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616167Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616168Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616169Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616170Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616172Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616173Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616174Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616175Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616176Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616177Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616178Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616179Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616185Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616187Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616216Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616217Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616218Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616219Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616220Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616221Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616222Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616223Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616224Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616225Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616226Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616227Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616228Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616229Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616230Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616231Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616232Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616233Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616234Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616235Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616236Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616237Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616238Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616239Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616241Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383616242Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383295201', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383295159', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383295117', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383295080', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383295037', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383294996', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383294879', None, None, None, None, None, '3717515') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 232 time used for this insertion : 0.892646074295044 save_final save missing photos in datou_result : time spend for datou_step_exec : 48.34951734542847 time spend to save output : 0.8958747386932373 total time spend for step 2 : 49.245392084121704 step3:rle_unique_nms_with_priority Sat Sep 13 08:52:14 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 75 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 8 nb_hashtags : 3 time to prepare the origin masks : 3.4833528995513916 time for calcul the mask position with numpy : 0.5774753093719482 nb_pixel_total : 7769404 time to create 1 rle with new method : 0.8259427547454834 time for calcul the mask position with numpy : 0.027584314346313477 nb_pixel_total : 106724 time to create 1 rle with old method : 0.11746883392333984 time for calcul the mask position with numpy : 0.024545907974243164 nb_pixel_total : 26572 time to create 1 rle with old method : 0.028789043426513672 time for calcul the mask position with numpy : 0.030079364776611328 nb_pixel_total : 72036 time to create 1 rle with old method : 0.07913374900817871 time for calcul the mask position with numpy : 0.0406031608581543 nb_pixel_total : 23447 time to create 1 rle with old method : 0.026378631591796875 time for calcul the mask position with numpy : 0.04478287696838379 nb_pixel_total : 110066 time to create 1 rle with old method : 0.11797118186950684 time for calcul the mask position with numpy : 0.03888130187988281 nb_pixel_total : 66183 time to create 1 rle with old method : 0.07123827934265137 time for calcul the mask position with numpy : 0.04067373275756836 nb_pixel_total : 55903 time to create 1 rle with old method : 0.05966305732727051 time for calcul the mask position with numpy : 0.029006004333496094 nb_pixel_total : 64065 time to create 1 rle with old method : 0.06935477256774902 create new chi : 2.2863729000091553 time to delete rle : 0.02185511589050293 batch 1 Loaded 17 chid ids of type : 3594 ++++++++++++++++Number RLEs to save : 8406 TO DO : save crop sub photo not yet done ! save time : 0.776116132736206 nb_obj : 16 nb_hashtags : 3 time to prepare the origin masks : 7.84033203125 time for calcul the mask position with numpy : 0.7050848007202148 nb_pixel_total : 7484823 time to create 1 rle with new method : 1.2151103019714355 time for calcul the mask position with numpy : 0.039758920669555664 nb_pixel_total : 29801 time to create 1 rle with old method : 0.03303885459899902 time for calcul the mask position with numpy : 0.040112972259521484 nb_pixel_total : 90837 time to create 1 rle with old method : 0.09973382949829102 time for calcul the mask position with numpy : 0.04015088081359863 nb_pixel_total : 19331 time to create 1 rle with old method : 0.021798372268676758 time for calcul the mask position with numpy : 0.039099931716918945 nb_pixel_total : 78999 time to create 1 rle with old method : 0.08948731422424316 time for calcul the mask position with numpy : 0.031835079193115234 nb_pixel_total : 9999 time to create 1 rle with old method : 0.011171817779541016 time for calcul the mask position with numpy : 0.030466079711914062 nb_pixel_total : 24621 time to create 1 rle with old method : 0.027783632278442383 time for calcul the mask position with numpy : 0.0272371768951416 nb_pixel_total : 36853 time to create 1 rle with old method : 0.04116415977478027 time for calcul the mask position with numpy : 0.02652454376220703 nb_pixel_total : 22757 time to create 1 rle with old method : 0.025487184524536133 time for calcul the mask position with numpy : 0.028045177459716797 nb_pixel_total : 171779 time to create 1 rle with new method : 0.7946946620941162 time for calcul the mask position with numpy : 0.04547834396362305 nb_pixel_total : 50802 time to create 1 rle with old method : 0.05662393569946289 time for calcul the mask position with numpy : 0.04591107368469238 nb_pixel_total : 14284 time to create 1 rle with old method : 0.020376920700073242 time for calcul the mask position with numpy : 0.04874014854431152 nb_pixel_total : 38700 time to create 1 rle with old method : 0.04313302040100098 time for calcul the mask position with numpy : 0.04038238525390625 nb_pixel_total : 96028 time to create 1 rle with old method : 0.10879969596862793 time for calcul the mask position with numpy : 0.04163408279418945 nb_pixel_total : 49127 time to create 1 rle with old method : 0.056496620178222656 time for calcul the mask position with numpy : 0.039174795150756836 nb_pixel_total : 30352 time to create 1 rle with old method : 0.03444480895996094 time for calcul the mask position with numpy : 0.03502178192138672 nb_pixel_total : 45307 time to create 1 rle with old method : 0.051694393157958984 create new chi : 4.1080052852630615 time to delete rle : 0.0023276805877685547 batch 1 Loaded 33 chid ids of type : 3594 ++++++++++++++++Number RLEs to save : 11316 TO DO : save crop sub photo not yet done ! save time : 0.8503293991088867 nb_obj : 11 nb_hashtags : 3 time to prepare the origin masks : 5.710904598236084 time for calcul the mask position with numpy : 0.752946138381958 nb_pixel_total : 7707847 time to create 1 rle with new method : 1.6033735275268555 time for calcul the mask position with numpy : 0.025341033935546875 nb_pixel_total : 23483 time to create 1 rle with old method : 0.027196645736694336 time for calcul the mask position with numpy : 0.027819156646728516 nb_pixel_total : 121710 time to create 1 rle with old method : 0.13712024688720703 time for calcul the mask position with numpy : 0.02777242660522461 nb_pixel_total : 61818 time to create 1 rle with old method : 0.07109379768371582 time for calcul the mask position with numpy : 0.027341604232788086 nb_pixel_total : 68364 time to create 1 rle with old method : 0.08014845848083496 time for calcul the mask position with numpy : 0.03171372413635254 nb_pixel_total : 13961 time to create 1 rle with old method : 0.01687908172607422 time for calcul the mask position with numpy : 0.027378082275390625 nb_pixel_total : 99644 time to create 1 rle with old method : 0.1115574836730957 time for calcul the mask position with numpy : 0.024960756301879883 nb_pixel_total : 53036 time to create 1 rle with old method : 0.06005454063415527 time for calcul the mask position with numpy : 0.025381803512573242 nb_pixel_total : 22943 time to create 1 rle with old method : 0.026163339614868164 time for calcul the mask position with numpy : 0.02681756019592285 nb_pixel_total : 77092 time to create 1 rle with old method : 0.08516836166381836 time for calcul the mask position with numpy : 0.026066303253173828 nb_pixel_total : 21823 time to create 1 rle with old method : 0.02481985092163086 time for calcul the mask position with numpy : 0.026808738708496094 nb_pixel_total : 22679 time to create 1 rle with old method : 0.02558159828186035 create new chi : 3.364967107772827 time to delete rle : 0.002061605453491211 batch 1 Loaded 23 chid ids of type : 3594 +++++++++++Number RLEs to save : 8746 TO DO : save crop sub photo not yet done ! save time : 0.8020436763763428 nb_obj : 14 nb_hashtags : 4 time to prepare the origin masks : 6.903714418411255 time for calcul the mask position with numpy : 0.4711792469024658 nb_pixel_total : 7028640 time to create 1 rle with new method : 0.6711363792419434 time for calcul the mask position with numpy : 0.04029703140258789 nb_pixel_total : 37085 time to create 1 rle with old method : 0.04098987579345703 time for calcul the mask position with numpy : 0.0338587760925293 nb_pixel_total : 8319 time to create 1 rle with old method : 0.009735584259033203 time for calcul the mask position with numpy : 0.02720165252685547 nb_pixel_total : 34776 time to create 1 rle with old method : 0.037255048751831055 time for calcul the mask position with numpy : 0.025372982025146484 nb_pixel_total : 41338 time to create 1 rle with old method : 0.04621171951293945 time for calcul the mask position with numpy : 0.025544404983520508 nb_pixel_total : 127731 time to create 1 rle with old method : 0.13897275924682617 time for calcul the mask position with numpy : 0.025679826736450195 nb_pixel_total : 142841 time to create 1 rle with old method : 0.15503144264221191 time for calcul the mask position with numpy : 0.025090694427490234 nb_pixel_total : 133757 time to create 1 rle with old method : 0.14767670631408691 time for calcul the mask position with numpy : 0.02557682991027832 nb_pixel_total : 10561 time to create 1 rle with old method : 0.011713504791259766 time for calcul the mask position with numpy : 0.024910926818847656 nb_pixel_total : 189955 time to create 1 rle with new method : 0.6516995429992676 time for calcul the mask position with numpy : 0.026941776275634766 nb_pixel_total : 123579 time to create 1 rle with old method : 0.13897228240966797 time for calcul the mask position with numpy : 0.027411937713623047 nb_pixel_total : 193916 time to create 1 rle with new method : 1.3676438331604004 time for calcul the mask position with numpy : 0.026392698287963867 nb_pixel_total : 112238 time to create 1 rle with old method : 0.12427186965942383 time for calcul the mask position with numpy : 0.025621652603149414 nb_pixel_total : 45191 time to create 1 rle with old method : 0.048642873764038086 time for calcul the mask position with numpy : 0.025537490844726562 nb_pixel_total : 64473 time to create 1 rle with old method : 0.07176566123962402 create new chi : 4.6197943687438965 time to delete rle : 0.0016074180603027344 batch 1 Loaded 29 chid ids of type : 3594 ++++++++++++++++++++Number RLEs to save : 13032 TO DO : save crop sub photo not yet done ! save time : 1.0775792598724365 nb_obj : 9 nb_hashtags : 5 time to prepare the origin masks : 5.7062928676605225 time for calcul the mask position with numpy : 0.4463672637939453 nb_pixel_total : 7828897 time to create 1 rle with new method : 0.7157189846038818 time for calcul the mask position with numpy : 0.024628877639770508 nb_pixel_total : 42136 time to create 1 rle with old method : 0.06046271324157715 time for calcul the mask position with numpy : 0.02507185935974121 nb_pixel_total : 29643 time to create 1 rle with old method : 0.03209686279296875 time for calcul the mask position with numpy : 0.023756980895996094 nb_pixel_total : 83932 time to create 1 rle with old method : 0.09014296531677246 time for calcul the mask position with numpy : 0.024233579635620117 nb_pixel_total : 43166 time to create 1 rle with old method : 0.04684019088745117 time for calcul the mask position with numpy : 0.023588180541992188 nb_pixel_total : 13842 time to create 1 rle with old method : 0.015047311782836914 time for calcul the mask position with numpy : 0.02388596534729004 nb_pixel_total : 50092 time to create 1 rle with old method : 0.05327653884887695 time for calcul the mask position with numpy : 0.0238039493560791 nb_pixel_total : 16304 time to create 1 rle with old method : 0.018021345138549805 time for calcul the mask position with numpy : 0.02815413475036621 nb_pixel_total : 105714 time to create 1 rle with old method : 0.13240814208984375 time for calcul the mask position with numpy : 0.024095535278320312 nb_pixel_total : 80674 time to create 1 rle with old method : 0.08735203742980957 create new chi : 1.9625611305236816 time to delete rle : 0.0010800361633300781 batch 1 Loaded 19 chid ids of type : 3594 +++++++++Number RLEs to save : 6958 TO DO : save crop sub photo not yet done ! save time : 0.6092331409454346 nb_obj : 8 nb_hashtags : 4 time to prepare the origin masks : 4.1886396408081055 time for calcul the mask position with numpy : 0.9877490997314453 nb_pixel_total : 7966373 time to create 1 rle with new method : 0.7933671474456787 time for calcul the mask position with numpy : 0.028474807739257812 nb_pixel_total : 31094 time to create 1 rle with old method : 0.03506731986999512 time for calcul the mask position with numpy : 0.02519702911376953 nb_pixel_total : 52243 time to create 1 rle with old method : 0.06013607978820801 time for calcul the mask position with numpy : 0.026337385177612305 nb_pixel_total : 13727 time to create 1 rle with old method : 0.015019893646240234 time for calcul the mask position with numpy : 0.02444744110107422 nb_pixel_total : 40214 time to create 1 rle with old method : 0.04690980911254883 time for calcul the mask position with numpy : 0.028486013412475586 nb_pixel_total : 95408 time to create 1 rle with old method : 0.11771988868713379 time for calcul the mask position with numpy : 0.025897502899169922 nb_pixel_total : 17706 time to create 1 rle with old method : 0.02037501335144043 time for calcul the mask position with numpy : 0.026706457138061523 nb_pixel_total : 17978 time to create 1 rle with old method : 0.01974630355834961 time for calcul the mask position with numpy : 0.025326967239379883 nb_pixel_total : 59657 time to create 1 rle with old method : 0.06583309173583984 create new chi : 2.4165446758270264 time to delete rle : 0.0008857250213623047 batch 1 Loaded 17 chid ids of type : 3594 ++++++++Number RLEs to save : 5728 TO DO : save crop sub photo not yet done ! save time : 0.5314180850982666 nb_obj : 9 nb_hashtags : 2 time to prepare the origin masks : 3.6256701946258545 time for calcul the mask position with numpy : 0.6163473129272461 nb_pixel_total : 7785097 time to create 1 rle with new method : 0.7578001022338867 time for calcul the mask position with numpy : 0.02496647834777832 nb_pixel_total : 58806 time to create 1 rle with old method : 0.061429738998413086 time for calcul the mask position with numpy : 0.03953814506530762 nb_pixel_total : 91317 time to create 1 rle with old method : 0.09670305252075195 time for calcul the mask position with numpy : 0.02483057975769043 nb_pixel_total : 15019 time to create 1 rle with old method : 0.01656937599182129 time for calcul the mask position with numpy : 0.026960372924804688 nb_pixel_total : 19069 time to create 1 rle with old method : 0.022879600524902344 time for calcul the mask position with numpy : 0.026181936264038086 nb_pixel_total : 30174 time to create 1 rle with old method : 0.04883909225463867 time for calcul the mask position with numpy : 0.025803089141845703 nb_pixel_total : 28902 time to create 1 rle with old method : 0.045668840408325195 time for calcul the mask position with numpy : 0.03345012664794922 nb_pixel_total : 94963 time to create 1 rle with old method : 0.10089254379272461 time for calcul the mask position with numpy : 0.026442527770996094 nb_pixel_total : 103553 time to create 1 rle with old method : 0.11614155769348145 time for calcul the mask position with numpy : 0.03043675422668457 nb_pixel_total : 67500 time to create 1 rle with old method : 0.07640576362609863 create new chi : 2.2593319416046143 time to delete rle : 0.0011456012725830078 batch 1 Loaded 19 chid ids of type : 3594 +++++++++++Number RLEs to save : 7238 TO DO : save crop sub photo not yet done ! save time : 1.3110709190368652 map_output_result : {1383295201: (0.0, 'Should be the crop_list due to order', 0), 1383295159: (0.0, 'Should be the crop_list due to order', 0), 1383295117: (0.0, 'Should be the crop_list due to order', 0), 1383295080: (0.0, 'Should be the crop_list due to order', 0), 1383295037: (0.0, 'Should be the crop_list due to order', 0), 1383294996: (0.0, 'Should be the crop_list due to order', 0), 1383294879: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1383295201, 1383295159, 1383295117, 1383295080, 1383295037, 1383294996, 1383294879] Looping around the photos to save general results len do output : 7 /1383295201.Didn't retrieve data . /1383295159.Didn't retrieve data . /1383295117.Didn't retrieve data . /1383295080.Didn't retrieve data . /1383295037.Didn't retrieve data . /1383294996.Didn't retrieve data . /1383294879.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383295201', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383295159', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383295117', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383295080', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383295037', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383294996', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383294879', None, None, None, None, None, '3717515') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 21 time used for this insertion : 0.012156963348388672 save_final save missing photos in datou_result : time spend for datou_step_exec : 65.91276001930237 time spend to save output : 0.012505054473876953 total time spend for step 3 : 65.92526507377625 step4:ventilate_hashtags_in_portfolio Sat Sep 13 08:53:20 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 26824390 get user id for portfolio 26824390 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26824390 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('carton','autre','pet_fonce','background','mal_croppe','metal','papier','environnement','flou','pet_clair','pehd')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26824390 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('carton','autre','pet_fonce','background','mal_croppe','metal','papier','environnement','flou','pet_clair','pehd')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26824390 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('carton','autre','pet_fonce','background','mal_croppe','metal','papier','environnement','flou','pet_clair','pehd')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://marlene.fotonower.com/velours/26824449,26824450,26824451,26824452,26824453,26824454,26824455,26824456,26824457,26824458,26824459?tags=carton,autre,pet_fonce,background,mal_croppe,metal,papier,environnement,flou,pet_clair,pehd Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1383295201, 1383295159, 1383295117, 1383295080, 1383295037, 1383294996, 1383294879] Looping around the photos to save general results len do output : 1 /26824390. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383295201', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383295159', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383295117', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383295080', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383295037', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383294996', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383294879', None, None, None, None, None, '3717515') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 8 time used for this insertion : 0.015055179595947266 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.9200024604797363 time spend to save output : 0.015299320220947266 total time spend for step 4 : 1.9353017807006836 step5:final Sat Sep 13 08:53:22 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1383295201: ('0.0773278873181217',), 1383295159: ('0.0773278873181217',), 1383295117: ('0.0773278873181217',), 1383295080: ('0.0773278873181217',), 1383295037: ('0.0773278873181217',), 1383294996: ('0.0773278873181217',), 1383294879: ('0.0773278873181217',)} new output for save of step final : {1383295201: ('0.0773278873181217',), 1383295159: ('0.0773278873181217',), 1383295117: ('0.0773278873181217',), 1383295080: ('0.0773278873181217',), 1383295037: ('0.0773278873181217',), 1383294996: ('0.0773278873181217',), 1383294879: ('0.0773278873181217',)} [1383295201, 1383295159, 1383295117, 1383295080, 1383295037, 1383294996, 1383294879] Looping around the photos to save general results len do output : 7 /1383295201.Didn't retrieve data . /1383295159.Didn't retrieve data . /1383295117.Didn't retrieve data . /1383295080.Didn't retrieve data . /1383295037.Didn't retrieve data . /1383294996.Didn't retrieve data . /1383294879.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383295201', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383295159', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383295117', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383295080', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383295037', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383294996', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383294879', None, None, None, None, None, '3717515') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 21 time used for this insertion : 0.012532711029052734 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.15479826927185059 time spend to save output : 0.012845277786254883 total time spend for step 5 : 0.16764354705810547 step6:blur_detection Sat Sep 13 08:53:22 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1757746227_1414272_1383295201_001c02754ebefba49d4bbe06a2e4e604.jpg resize: (2160, 3840) 1383295201 -6.698333555271765 treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda.jpg resize: (2160, 3840) 1383295159 -6.817668224901485 treat image : temp/1757746227_1414272_1383295117_5e836c685622fe6787d74a082f8af2ea.jpg resize: (2160, 3840) 1383295117 -6.846964678927318 treat image : temp/1757746227_1414272_1383295080_a992b2f296cb8cb7005ad07452656182.jpg resize: (2160, 3840) 1383295080 -6.92395389681924 treat image : temp/1757746227_1414272_1383295037_1376fc2387eb78bba3652d66a48f9192.jpg resize: (2160, 3840) 1383295037 -6.176278048868428 treat image : temp/1757746227_1414272_1383294996_944f8e787f62c60154899accdca4f40e.jpg resize: (2160, 3840) 1383294996 -6.7483106955569365 treat image : temp/1757746227_1414272_1383294879_e811120f6cdf6e191a68c92a438bd123.jpg resize: (2160, 3840) 1383294879 -6.801818380071483 treat image : temp/1757746227_1414272_1383295201_001c02754ebefba49d4bbe06a2e4e604_rle_crop_3957971319_0.png resize: (271, 280) 1383616130 -3.7653864165143562 treat image : temp/1757746227_1414272_1383295201_001c02754ebefba49d4bbe06a2e4e604_rle_crop_3957971322_0.png resize: (220, 149) 1383616131 -1.4263454464670722 treat image : temp/1757746227_1414272_1383295201_001c02754ebefba49d4bbe06a2e4e604_rle_crop_3957971323_0.png resize: (529, 309) 1383616132 -3.876781123649635 treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971326_0.png resize: (317, 263) 1383616133 -3.4032570745717807 treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971327_0.png resize: (221, 213) 1383616134 -3.6427944437367095 treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971330_0.png resize: (274, 222) 1383616135 -2.135106722968958 treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971331_0.png resize: (142, 139) 1383616136 -2.6842349230866467 treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971332_0.png resize: (299, 239) 1383616137 -5.038288382540011 treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971333_0.png resize: (603, 381) 1383616138 -4.906398049542453 treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971336_0.png resize: (189, 197) 1383616139 -3.5896188544956265 treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971337_0.png resize: (91, 157) 1383616140 -3.9090920105848364 treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971338_0.png resize: (438, 274) 1383616141 -3.624766301206296 treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971339_0.png resize: (179, 181) 1383616142 -2.997849635039733 treat image : temp/1757746227_1414272_1383295117_5e836c685622fe6787d74a082f8af2ea_rle_crop_3957971342_0.png resize: (196, 181) 1383616143 -0.6431682607145032 treat image : temp/1757746227_1414272_1383295117_5e836c685622fe6787d74a082f8af2ea_rle_crop_3957971343_0.png resize: (247, 156) 1383616144 -3.730969095773568 treat image : temp/1757746227_1414272_1383295117_5e836c685622fe6787d74a082f8af2ea_rle_crop_3957971345_0.png resize: (183, 155) 1383616145 -3.1187961780578464 treat image : temp/1757746227_1414272_1383295117_5e836c685622fe6787d74a082f8af2ea_rle_crop_3957971346_0.png resize: (326, 224) 1383616146 -4.767561594335475 treat image : temp/1757746227_1414272_1383295117_5e836c685622fe6787d74a082f8af2ea_rle_crop_3957971349_0.png resize: (377, 356) 1383616147 -4.725649643634044 treat image : temp/1757746227_1414272_1383295080_a992b2f296cb8cb7005ad07452656182_rle_crop_3957971353_0.png resize: (302, 339) 1383616148 -2.010605637309666 treat image : temp/1757746227_1414272_1383295080_a992b2f296cb8cb7005ad07452656182_rle_crop_3957971354_0.png resize: (207, 257) 1383616149 -3.673998390381732 treat image : temp/1757746227_1414272_1383295080_a992b2f296cb8cb7005ad07452656182_rle_crop_3957971356_0.png resize: (570, 582) 1383616151 -1.7991452769500487 treat image : temp/1757746227_1414272_1383295080_a992b2f296cb8cb7005ad07452656182_rle_crop_3957971357_0.png resize: (512, 405) 1383616152 -4.543992516456759 treat image : temp/1757746227_1414272_1383295080_a992b2f296cb8cb7005ad07452656182_rle_crop_3957971359_0.png resize: (188, 115) 1383616153 -3.545279299854385 treat image : temp/1757746227_1414272_1383295080_a992b2f296cb8cb7005ad07452656182_rle_crop_3957971363_0.png resize: (374, 435) 1383616154 -4.236721695701339 treat image : temp/1757746227_1414272_1383295080_a992b2f296cb8cb7005ad07452656182_rle_crop_3957971364_0.png resize: (310, 177) 1383616156 -4.10537111098909 treat image : temp/1757746227_1414272_1383295080_a992b2f296cb8cb7005ad07452656182_rle_crop_3957971365_0.png resize: (429, 484) 1383616157 -4.552123349069582 treat image : temp/1757746227_1414272_1383295037_1376fc2387eb78bba3652d66a48f9192_rle_crop_3957971367_0.png resize: (394, 250) 1383616158 -3.668015083175239 treat image : temp/1757746227_1414272_1383295037_1376fc2387eb78bba3652d66a48f9192_rle_crop_3957971370_0.png resize: (291, 286) 1383616159 -2.23500502286443 treat image : temp/1757746227_1414272_1383294996_944f8e787f62c60154899accdca4f40e_rle_crop_3957971377_0.png resize: (218, 153) 1383616160 -0.48085928713352766 treat image : temp/1757746227_1414272_1383294996_944f8e787f62c60154899accdca4f40e_rle_crop_3957971378_0.png resize: (140, 160) 1383616161 -3.500329902890524 treat image : temp/1757746227_1414272_1383294996_944f8e787f62c60154899accdca4f40e_rle_crop_3957971379_0.png resize: (272, 492) 1383616162 -3.690453285322243 treat image : temp/1757746227_1414272_1383294996_944f8e787f62c60154899accdca4f40e_rle_crop_3957971381_0.png resize: (137, 116) 1383616163 -2.119002239666961 treat image : temp/1757746227_1414272_1383294996_944f8e787f62c60154899accdca4f40e_rle_crop_3957971383_0.png resize: (195, 189) 1383616164 -3.9297819481769722 treat image : temp/1757746227_1414272_1383294879_e811120f6cdf6e191a68c92a438bd123_rle_crop_3957971384_0.png resize: (303, 356) 1383616165 -3.890131020960916 treat image : temp/1757746227_1414272_1383294879_e811120f6cdf6e191a68c92a438bd123_rle_crop_3957971387_0.png resize: (365, 131) 1383616166 -2.1766054216916046 treat image : temp/1757746227_1414272_1383294879_e811120f6cdf6e191a68c92a438bd123_rle_crop_3957971389_0.png resize: (187, 142) 1383616167 -4.261346341833948 treat image : temp/1757746227_1414272_1383294879_e811120f6cdf6e191a68c92a438bd123_rle_crop_3957971390_0.png resize: (144, 159) 1383616168 -3.766247740709869 treat image : temp/1757746227_1414272_1383294879_e811120f6cdf6e191a68c92a438bd123_rle_crop_3957971391_0.png resize: (417, 290) 1383616169 -5.2687445580466115 treat image : temp/1757746227_1414272_1383294879_e811120f6cdf6e191a68c92a438bd123_rle_crop_3957971392_0.png resize: (287, 275) 1383616170 -4.2484597453338795 treat image : temp/1757746227_1414272_1383295201_001c02754ebefba49d4bbe06a2e4e604_rle_crop_3957971324_0.png resize: (248, 145) 1383616172 -3.5458909958523517 treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971335_0.png resize: (204, 240) 1383616173 -4.065986344155811 treat image : temp/1757746227_1414272_1383295117_5e836c685622fe6787d74a082f8af2ea_rle_crop_3957971348_0.png resize: (150, 141) 1383616174 -2.8627139042139045 treat image : temp/1757746227_1414272_1383295117_5e836c685622fe6787d74a082f8af2ea_rle_crop_3957971350_0.png resize: (415, 251) 1383616175 -3.203077585430895 treat image : temp/1757746227_1414272_1383295080_a992b2f296cb8cb7005ad07452656182_rle_crop_3957971358_0.png resize: (736, 392) 1383616176 -5.133584634526433 treat image : temp/1757746227_1414272_1383295037_1376fc2387eb78bba3652d66a48f9192_rle_crop_3957971369_0.png resize: (114, 198) 1383616177 -2.0748000059570755 treat image : temp/1757746227_1414272_1383295037_1376fc2387eb78bba3652d66a48f9192_rle_crop_3957971374_0.png resize: (246, 182) 1383616178 -2.7898222329594478 treat image : temp/1757746227_1414272_1383294996_944f8e787f62c60154899accdca4f40e_rle_crop_3957971382_0.png resize: (247, 278) 1383616179 -2.789777041294677 treat image : temp/1757746227_1414272_1383295080_a992b2f296cb8cb7005ad07452656182_rle_crop_3957971366_0.png resize: (221, 257) 1383616185 -4.0979089608728705 treat image : temp/1757746227_1414272_1383295037_1376fc2387eb78bba3652d66a48f9192_rle_crop_3957971371_0.png resize: (131, 131) 1383616187 -3.9244684956605624 treat image : temp/1757746227_1414272_1383295201_001c02754ebefba49d4bbe06a2e4e604_rle_crop_3957971318_0.png resize: (189, 517) 1383616216 -3.489588675421265 treat image : temp/1757746227_1414272_1383295201_001c02754ebefba49d4bbe06a2e4e604_rle_crop_3957971320_0.png resize: (369, 250) 1383616217 -4.9958846287640775 treat image : temp/1757746227_1414272_1383295201_001c02754ebefba49d4bbe06a2e4e604_rle_crop_3957971321_0.png resize: (401, 340) 1383616218 -4.921579765281123 treat image : temp/1757746227_1414272_1383295201_001c02754ebefba49d4bbe06a2e4e604_rle_crop_3957971325_0.png resize: (442, 400) 1383616219 -4.608344327602495 treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971328_0.png resize: (263, 267) 1383616220 -4.701285544126175 treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971329_0.png resize: (430, 340) 1383616221 -2.890198204370614 treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971334_0.png resize: (141, 257) 1383616222 -2.9151941445091025 treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971340_0.png resize: (296, 562) 1383616223 -3.777570080141912 treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971341_0.png resize: (325, 134) 1383616224 -3.4785999445345106 treat image : temp/1757746227_1414272_1383295117_5e836c685622fe6787d74a082f8af2ea_rle_crop_3957971344_0.png resize: (380, 274) 1383616225 -3.5168373081807407 treat image : temp/1757746227_1414272_1383295117_5e836c685622fe6787d74a082f8af2ea_rle_crop_3957971347_0.png resize: (346, 400) 1383616226 -4.566935602635208 treat image : temp/1757746227_1414272_1383295117_5e836c685622fe6787d74a082f8af2ea_rle_crop_3957971351_0.png resize: (408, 484) 1383616227 -1.4902940403507459 treat image : temp/1757746227_1414272_1383295117_5e836c685622fe6787d74a082f8af2ea_rle_crop_3957971352_0.png resize: (161, 272) 1383616228 -3.2973644384105247 treat image : temp/1757746227_1414272_1383295080_a992b2f296cb8cb7005ad07452656182_rle_crop_3957971355_0.png resize: (319, 505) 1383616229 -4.1381033488657595 treat image : temp/1757746227_1414272_1383295080_a992b2f296cb8cb7005ad07452656182_rle_crop_3957971360_0.png resize: (342, 541) 1383616230 -4.242842969657956 treat image : temp/1757746227_1414272_1383295080_a992b2f296cb8cb7005ad07452656182_rle_crop_3957971361_0.png resize: (458, 407) 1383616231 -4.710633025924543 treat image : temp/1757746227_1414272_1383295080_a992b2f296cb8cb7005ad07452656182_rle_crop_3957971362_0.png resize: (420, 445) 1383616232 -4.83421939199515 treat image : temp/1757746227_1414272_1383295037_1376fc2387eb78bba3652d66a48f9192_rle_crop_3957971368_0.png resize: (447, 484) 1383616233 -3.76078017380885 treat image : temp/1757746227_1414272_1383295037_1376fc2387eb78bba3652d66a48f9192_rle_crop_3957971373_0.png resize: (327, 421) 1383616234 -3.654924565331567 treat image : temp/1757746227_1414272_1383295037_1376fc2387eb78bba3652d66a48f9192_rle_crop_3957971375_0.png resize: (273, 354) 1383616235 -3.2091352542148615 treat image : temp/1757746227_1414272_1383294996_944f8e787f62c60154899accdca4f40e_rle_crop_3957971376_0.png resize: (334, 235) 1383616236 -5.049223085679287 treat image : temp/1757746227_1414272_1383294879_e811120f6cdf6e191a68c92a438bd123_rle_crop_3957971385_0.png resize: (247, 523) 1383616237 -5.097740443428802 treat image : temp/1757746227_1414272_1383294879_e811120f6cdf6e191a68c92a438bd123_rle_crop_3957971386_0.png resize: (355, 342) 1383616238 -4.948305852130128 treat image : temp/1757746227_1414272_1383294879_e811120f6cdf6e191a68c92a438bd123_rle_crop_3957971388_0.png resize: (171, 248) 1383616239 -4.456918797959662 treat image : temp/1757746227_1414272_1383295037_1376fc2387eb78bba3652d66a48f9192_rle_crop_3957971372_0.png resize: (138, 423) 1383616241 -3.1069221524046595 treat image : temp/1757746227_1414272_1383294996_944f8e787f62c60154899accdca4f40e_rle_crop_3957971380_0.png resize: (221, 210) 1383616242 -1.9617016241291516 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 82 time used for this insertion : 0.0159304141998291 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 82 time used for this insertion : 0.35594701766967773 save missing photos in datou_result : time spend for datou_step_exec : 27.837390661239624 time spend to save output : 0.37664127349853516 total time spend for step 6 : 28.21403193473816 step7:brightness Sat Sep 13 08:53:51 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1757746227_1414272_1383295201_001c02754ebefba49d4bbe06a2e4e604.jpg treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda.jpg treat image : temp/1757746227_1414272_1383295117_5e836c685622fe6787d74a082f8af2ea.jpg treat image : temp/1757746227_1414272_1383295080_a992b2f296cb8cb7005ad07452656182.jpg treat image : temp/1757746227_1414272_1383295037_1376fc2387eb78bba3652d66a48f9192.jpg treat image : temp/1757746227_1414272_1383294996_944f8e787f62c60154899accdca4f40e.jpg treat image : temp/1757746227_1414272_1383294879_e811120f6cdf6e191a68c92a438bd123.jpg treat image : temp/1757746227_1414272_1383295201_001c02754ebefba49d4bbe06a2e4e604_rle_crop_3957971319_0.png treat image : temp/1757746227_1414272_1383295201_001c02754ebefba49d4bbe06a2e4e604_rle_crop_3957971322_0.png treat image : temp/1757746227_1414272_1383295201_001c02754ebefba49d4bbe06a2e4e604_rle_crop_3957971323_0.png treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971326_0.png treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971327_0.png treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971330_0.png treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971331_0.png treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971332_0.png treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971333_0.png treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971336_0.png treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971337_0.png treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971338_0.png treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971339_0.png treat image : temp/1757746227_1414272_1383295117_5e836c685622fe6787d74a082f8af2ea_rle_crop_3957971342_0.png treat image : temp/1757746227_1414272_1383295117_5e836c685622fe6787d74a082f8af2ea_rle_crop_3957971343_0.png treat image : temp/1757746227_1414272_1383295117_5e836c685622fe6787d74a082f8af2ea_rle_crop_3957971345_0.png treat image : temp/1757746227_1414272_1383295117_5e836c685622fe6787d74a082f8af2ea_rle_crop_3957971346_0.png treat image : temp/1757746227_1414272_1383295117_5e836c685622fe6787d74a082f8af2ea_rle_crop_3957971349_0.png treat image : temp/1757746227_1414272_1383295080_a992b2f296cb8cb7005ad07452656182_rle_crop_3957971353_0.png treat image : temp/1757746227_1414272_1383295080_a992b2f296cb8cb7005ad07452656182_rle_crop_3957971354_0.png treat image : temp/1757746227_1414272_1383295080_a992b2f296cb8cb7005ad07452656182_rle_crop_3957971356_0.png treat image : temp/1757746227_1414272_1383295080_a992b2f296cb8cb7005ad07452656182_rle_crop_3957971357_0.png treat image : temp/1757746227_1414272_1383295080_a992b2f296cb8cb7005ad07452656182_rle_crop_3957971359_0.png treat image : temp/1757746227_1414272_1383295080_a992b2f296cb8cb7005ad07452656182_rle_crop_3957971363_0.png treat image : temp/1757746227_1414272_1383295080_a992b2f296cb8cb7005ad07452656182_rle_crop_3957971364_0.png treat image : temp/1757746227_1414272_1383295080_a992b2f296cb8cb7005ad07452656182_rle_crop_3957971365_0.png treat image : temp/1757746227_1414272_1383295037_1376fc2387eb78bba3652d66a48f9192_rle_crop_3957971367_0.png treat image : temp/1757746227_1414272_1383295037_1376fc2387eb78bba3652d66a48f9192_rle_crop_3957971370_0.png treat image : temp/1757746227_1414272_1383294996_944f8e787f62c60154899accdca4f40e_rle_crop_3957971377_0.png treat image : temp/1757746227_1414272_1383294996_944f8e787f62c60154899accdca4f40e_rle_crop_3957971378_0.png treat image : temp/1757746227_1414272_1383294996_944f8e787f62c60154899accdca4f40e_rle_crop_3957971379_0.png treat image : temp/1757746227_1414272_1383294996_944f8e787f62c60154899accdca4f40e_rle_crop_3957971381_0.png treat image : temp/1757746227_1414272_1383294996_944f8e787f62c60154899accdca4f40e_rle_crop_3957971383_0.png treat image : temp/1757746227_1414272_1383294879_e811120f6cdf6e191a68c92a438bd123_rle_crop_3957971384_0.png treat image : temp/1757746227_1414272_1383294879_e811120f6cdf6e191a68c92a438bd123_rle_crop_3957971387_0.png treat image : temp/1757746227_1414272_1383294879_e811120f6cdf6e191a68c92a438bd123_rle_crop_3957971389_0.png treat image : temp/1757746227_1414272_1383294879_e811120f6cdf6e191a68c92a438bd123_rle_crop_3957971390_0.png treat image : temp/1757746227_1414272_1383294879_e811120f6cdf6e191a68c92a438bd123_rle_crop_3957971391_0.png treat image : temp/1757746227_1414272_1383294879_e811120f6cdf6e191a68c92a438bd123_rle_crop_3957971392_0.png treat image : temp/1757746227_1414272_1383295201_001c02754ebefba49d4bbe06a2e4e604_rle_crop_3957971324_0.png treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971335_0.png treat image : temp/1757746227_1414272_1383295117_5e836c685622fe6787d74a082f8af2ea_rle_crop_3957971348_0.png treat image : temp/1757746227_1414272_1383295117_5e836c685622fe6787d74a082f8af2ea_rle_crop_3957971350_0.png treat image : temp/1757746227_1414272_1383295080_a992b2f296cb8cb7005ad07452656182_rle_crop_3957971358_0.png treat image : temp/1757746227_1414272_1383295037_1376fc2387eb78bba3652d66a48f9192_rle_crop_3957971369_0.png treat image : temp/1757746227_1414272_1383295037_1376fc2387eb78bba3652d66a48f9192_rle_crop_3957971374_0.png treat image : temp/1757746227_1414272_1383294996_944f8e787f62c60154899accdca4f40e_rle_crop_3957971382_0.png treat image : temp/1757746227_1414272_1383295080_a992b2f296cb8cb7005ad07452656182_rle_crop_3957971366_0.png treat image : temp/1757746227_1414272_1383295037_1376fc2387eb78bba3652d66a48f9192_rle_crop_3957971371_0.png treat image : temp/1757746227_1414272_1383295201_001c02754ebefba49d4bbe06a2e4e604_rle_crop_3957971318_0.png treat image : temp/1757746227_1414272_1383295201_001c02754ebefba49d4bbe06a2e4e604_rle_crop_3957971320_0.png treat image : temp/1757746227_1414272_1383295201_001c02754ebefba49d4bbe06a2e4e604_rle_crop_3957971321_0.png treat image : temp/1757746227_1414272_1383295201_001c02754ebefba49d4bbe06a2e4e604_rle_crop_3957971325_0.png treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971328_0.png treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971329_0.png treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971334_0.png treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971340_0.png treat image : temp/1757746227_1414272_1383295159_8cf808f6975033c82d586bc761c0acda_rle_crop_3957971341_0.png treat image : temp/1757746227_1414272_1383295117_5e836c685622fe6787d74a082f8af2ea_rle_crop_3957971344_0.png treat image : temp/1757746227_1414272_1383295117_5e836c685622fe6787d74a082f8af2ea_rle_crop_3957971347_0.png treat image : temp/1757746227_1414272_1383295117_5e836c685622fe6787d74a082f8af2ea_rle_crop_3957971351_0.png treat image : temp/1757746227_1414272_1383295117_5e836c685622fe6787d74a082f8af2ea_rle_crop_3957971352_0.png treat image : temp/1757746227_1414272_1383295080_a992b2f296cb8cb7005ad07452656182_rle_crop_3957971355_0.png treat image : temp/1757746227_1414272_1383295080_a992b2f296cb8cb7005ad07452656182_rle_crop_3957971360_0.png treat image : temp/1757746227_1414272_1383295080_a992b2f296cb8cb7005ad07452656182_rle_crop_3957971361_0.png treat image : temp/1757746227_1414272_1383295080_a992b2f296cb8cb7005ad07452656182_rle_crop_3957971362_0.png treat image : temp/1757746227_1414272_1383295037_1376fc2387eb78bba3652d66a48f9192_rle_crop_3957971368_0.png treat image : temp/1757746227_1414272_1383295037_1376fc2387eb78bba3652d66a48f9192_rle_crop_3957971373_0.png treat image : temp/1757746227_1414272_1383295037_1376fc2387eb78bba3652d66a48f9192_rle_crop_3957971375_0.png treat image : temp/1757746227_1414272_1383294996_944f8e787f62c60154899accdca4f40e_rle_crop_3957971376_0.png treat image : temp/1757746227_1414272_1383294879_e811120f6cdf6e191a68c92a438bd123_rle_crop_3957971385_0.png treat image : temp/1757746227_1414272_1383294879_e811120f6cdf6e191a68c92a438bd123_rle_crop_3957971386_0.png treat image : temp/1757746227_1414272_1383294879_e811120f6cdf6e191a68c92a438bd123_rle_crop_3957971388_0.png treat image : temp/1757746227_1414272_1383295037_1376fc2387eb78bba3652d66a48f9192_rle_crop_3957971372_0.png treat image : temp/1757746227_1414272_1383294996_944f8e787f62c60154899accdca4f40e_rle_crop_3957971380_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 82 time used for this insertion : 0.01819300651550293 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 82 time used for this insertion : 0.24282479286193848 save missing photos in datou_result : time spend for datou_step_exec : 7.129548788070679 time spend to save output : 0.2655348777770996 total time spend for step 7 : 7.395083665847778 step8:velours_tree Sat Sep 13 08:53:58 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.20415592193603516 time spend to save output : 4.124641418457031e-05 total time spend for step 8 : 0.20419716835021973 step9:send_mail_cod Sat Sep 13 08:53:58 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P26824390_13-09-2025_08_53_58.pdf 26824449 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette268244491757746438 26824450 change filename to text .change filename to text .imagette268244501757746439 26824451 imagette268244511757746439 26824452 imagette268244521757746439 26824453 imagette268244531757746439 26824454 change filename to text .change filename to text .imagette268244541757746439 26824455 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette268244551757746439 26824457 imagette268244571757746441 26824458 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette268244581757746441 26824459 imagette268244591757746443 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=26824390 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://marlene.fotonower.com/velours/26824449,26824450,26824451,26824452,26824453,26824454,26824455,26824456,26824457,26824458,26824459?tags=carton,autre,pet_fonce,background,mal_croppe,metal,papier,environnement,flou,pet_clair,pehd args[1383295201] : ((1383295201, -6.698333555271765, 492609224), (1383295201, 0.7226978410539497, 2107752395), '0.0773278873181217') We are sending mail with results at report@fotonower.com args[1383295159] : ((1383295159, -6.817668224901485, 492609224), (1383295159, 0.6818376417191336, 2107752395), '0.0773278873181217') We are sending mail with results at report@fotonower.com args[1383295117] : ((1383295117, -6.846964678927318, 492609224), (1383295117, 0.7451213752376182, 2107752395), '0.0773278873181217') We are sending mail with results at report@fotonower.com args[1383295080] : ((1383295080, -6.92395389681924, 492609224), (1383295080, 0.7428291688902514, 2107752395), '0.0773278873181217') We are sending mail with results at report@fotonower.com args[1383295037] : ((1383295037, -6.176278048868428, 492609224), (1383295037, 0.564974664594312, 2107752395), '0.0773278873181217') We are sending mail with results at report@fotonower.com args[1383294996] : ((1383294996, -6.7483106955569365, 492609224), (1383294996, 0.6888995161325899, 2107752395), '0.0773278873181217') We are sending mail with results at report@fotonower.com args[1383294879] : ((1383294879, -6.801818380071483, 492609224), (1383294879, 0.6655833377629238, 2107752395), '0.0773278873181217') We are sending mail with results at report@fotonower.com refus_total : 0.0773278873181217 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=26824390 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26824390_13-09-2025_08_53_58.pdf results_Auto_P26824390_13-09-2025_08_53_58.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26824390_13-09-2025_08_53_58.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','26824390','results_Auto_P26824390_13-09-2025_08_53_58.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26824390_13-09-2025_08_53_58.pdf','pdf','','0.67','0.0773278873181217') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/26824390

https://www.fotonower.com/image?json=false&list_photos_id=1383295201
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1383295159
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1383295117
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1383295080
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1383295037
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1383294996
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1383294879
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 7.73%
Veuillez trouver les photos des contaminants.

exemples de contaminants: carton: https://www.fotonower.com/view/26824449?limit=200
exemples de contaminants: autre: https://www.fotonower.com/view/26824450?limit=200
exemples de contaminants: metal: https://www.fotonower.com/view/26824454?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/26824455?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/26824458?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26824390_13-09-2025_08_53_58.pdf.

Lien vers velours :https://marlene.fotonower.com/velours/26824449,26824450,26824451,26824452,26824453,26824454,26824455,26824456,26824457,26824458,26824459?tags=carton,autre,pet_fonce,background,mal_croppe,metal,papier,environnement,flou,pet_clair,pehd.


L'équipe Fotonower 202 b'' Server: nginx Date: Sat, 13 Sep 2025 06:54:06 GMT Content-Length: 0 Connection: close X-Message-Id: 9jCc1_e_TC2WqG7BYg7M5w Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1383295201, 1383295159, 1383295117, 1383295080, 1383295037, 1383294996, 1383294879] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383295201', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383295159', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383295117', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383295080', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383295037', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383294996', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383294879', None, None, None, None, None, '3717515') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 7 time used for this insertion : 0.044867753982543945 save_final save missing photos in datou_result : time spend for datou_step_exec : 7.690305471420288 time spend to save output : 0.045079946517944336 total time spend for step 9 : 7.735385417938232 step10:split_time_score Sat Sep 13 08:54:06 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('18', 7),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 11092025 26824390 Nombre de photos uploadées : 7 / 23040 (0%) 11092025 26824390 Nombre de photos taguées (types de déchets): 0 / 7 (0%) 11092025 26824390 Nombre de photos taguées (volume) : 0 / 7 (0%) elapsed_time : load_data_split_time_score 3.5762786865234375e-06 elapsed_time : order_list_meta_photo_and_scores 1.2159347534179688e-05 ??????? elapsed_time : fill_and_build_computed_from_old_data 0.00048160552978515625 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.20305633544921875 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.023175687631709546 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26747097_11-09-2025_07_23_16.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26747097 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26747097 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26824345 order by id desc limit 1 Qualite : 0.10983450681584361 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26758237_11-09-2025_11_44_13.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26758237 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26758237 AND mptpi.`type`=3594 To do Qualite : 0.020712950646425057 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26759506_11-09-2025_12_13_16.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26759506 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26759506 AND mptpi.`type`=3726 To do Qualite : 0.10079201630015432 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26761746_11-09-2025_18_07_06.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26761746 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26761746 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26824368 order by id desc limit 1 Qualite : 0.101249112654321 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26766059_11-09-2025_22_05_30.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26766059 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26766059 AND mptpi.`type`=3594 To do Qualite : 0.0773278873181217 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26824390_13-09-2025_08_53_58.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26824390 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26824390 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26824393 order by id desc limit 1 NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'11092025': {'nb_upload': 7, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1383295201, 1383295159, 1383295117, 1383295080, 1383295037, 1383294996, 1383294879] Looping around the photos to save general results len do output : 1 /26824390Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383295201', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383295159', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383295117', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383295080', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383295037', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383294996', None, None, None, None, None, '3717515') ('3318', None, None, None, None, None, None, None, '3717515') ('3318', '26824390', '1383294879', None, None, None, None, None, '3717515') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 8 time used for this insertion : 0.05021333694458008 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.062198877334595 time spend to save output : 0.05051589012145996 total time spend for step 10 : 4.112714767456055 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 7 set_done_treatment 96.17user 87.65system 3:45.84elapsed 81%CPU (0avgtext+0avgdata 3912164maxresident)k 473576inputs+103536outputs (17180major+9375382minor)pagefaults 0swaps