python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 1264403 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['3555385'] with mtr_portfolio_ids : ['26070739'] and first list_photo_ids : [] new path : /proc/1264403/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 23 ; length of list_pids : 23 ; length of list_args : 23 time to download the photos : 2.842472791671753 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Wed Aug 20 08:20:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10393 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-08-20 08:20:33.949367: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-08-20 08:20:33.976497: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-08-20 08:20:33.978510: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f6b6c000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-08-20 08:20:33.978572: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-08-20 08:20:33.982398: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-08-20 08:20:34.097862: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x38142e40 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-08-20 08:20:34.097908: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-08-20 08:20:34.099258: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-20 08:20:34.099654: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-20 08:20:34.102726: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-20 08:20:34.105663: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-20 08:20:34.106052: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-20 08:20:34.108751: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-20 08:20:34.110093: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-20 08:20:34.115228: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-20 08:20:34.116682: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-20 08:20:34.116761: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-20 08:20:34.117513: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-08-20 08:20:34.117528: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-08-20 08:20:34.117538: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-08-20 08:20:34.118835: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9483 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-08-20 08:20:34.403360: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-20 08:20:34.403461: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-20 08:20:34.403484: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-20 08:20:34.403505: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-20 08:20:34.403524: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-20 08:20:34.403543: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-20 08:20:34.403562: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-20 08:20:34.403581: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-20 08:20:34.408332: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-20 08:20:34.410218: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-20 08:20:34.410276: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-20 08:20:34.410301: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-20 08:20:34.410324: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-20 08:20:34.410346: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-20 08:20:34.410367: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-20 08:20:34.410388: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-20 08:20:34.410411: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-20 08:20:34.412216: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-20 08:20:34.412255: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-08-20 08:20:34.412266: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-08-20 08:20:34.412276: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-08-20 08:20:34.413970: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9483 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-08-20 08:20:43.767804: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-20 08:20:43.992290: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 23 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 30.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 2 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 20.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 2 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 24.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 16.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 29.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 26.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 24.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 34.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 18.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 31.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 25.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 26.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 21.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 18.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 21.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 22.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 23.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 24.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 20.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 24.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 14 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 30.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 22.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 25.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 Detection mask done ! Trying to reset tf kernel 1264970 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 134 tf kernel not reseted sub process len(results) : 23 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 23 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5423 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.00022029876708984375 nb_pixel_total : 4380 time to create 1 rle with old method : 0.006211519241333008 length of segment : 86 time for calcul the mask position with numpy : 0.0002694129943847656 nb_pixel_total : 11623 time to create 1 rle with old method : 0.013640403747558594 length of segment : 120 time for calcul the mask position with numpy : 0.03214287757873535 nb_pixel_total : 765380 time to create 1 rle with new method : 0.10823678970336914 length of segment : 995 time for calcul the mask position with numpy : 0.002184152603149414 nb_pixel_total : 113791 time to create 1 rle with old method : 0.12677454948425293 length of segment : 613 time for calcul the mask position with numpy : 0.015461921691894531 nb_pixel_total : 791762 time to create 1 rle with new method : 0.030599594116210938 length of segment : 993 time for calcul the mask position with numpy : 0.00020837783813476562 nb_pixel_total : 7651 time to create 1 rle with old method : 0.009146690368652344 length of segment : 117 time for calcul the mask position with numpy : 0.00010061264038085938 nb_pixel_total : 2248 time to create 1 rle with old method : 0.0028421878814697266 length of segment : 72 time for calcul the mask position with numpy : 0.00023746490478515625 nb_pixel_total : 11510 time to create 1 rle with old method : 0.013785362243652344 length of segment : 155 time for calcul the mask position with numpy : 0.00038123130798339844 nb_pixel_total : 17841 time to create 1 rle with old method : 0.021670103073120117 length of segment : 206 time for calcul the mask position with numpy : 0.022817373275756836 nb_pixel_total : 1453848 time to create 1 rle with new method : 0.1638813018798828 length of segment : 1294 time for calcul the mask position with numpy : 0.0008389949798583984 nb_pixel_total : 19711 time to create 1 rle with old method : 0.023464441299438477 length of segment : 306 time for calcul the mask position with numpy : 0.00021648406982421875 nb_pixel_total : 2690 time to create 1 rle with old method : 0.0033500194549560547 length of segment : 45 time for calcul the mask position with numpy : 0.0250246524810791 nb_pixel_total : 720260 time to create 1 rle with new method : 0.049985647201538086 length of segment : 932 time for calcul the mask position with numpy : 0.0007615089416503906 nb_pixel_total : 10663 time to create 1 rle with old method : 0.013494253158569336 length of segment : 185 time for calcul the mask position with numpy : 0.0003707408905029297 nb_pixel_total : 12262 time to create 1 rle with old method : 0.014470577239990234 length of segment : 153 time for calcul the mask position with numpy : 0.00019407272338867188 nb_pixel_total : 8082 time to create 1 rle with old method : 0.009523391723632812 length of segment : 117 time for calcul the mask position with numpy : 0.00017452239990234375 nb_pixel_total : 2255 time to create 1 rle with old method : 0.0028409957885742188 length of segment : 65 time for calcul the mask position with numpy : 0.0006787776947021484 nb_pixel_total : 19387 time to create 1 rle with old method : 0.02292037010192871 length of segment : 204 time for calcul the mask position with numpy : 0.0010077953338623047 nb_pixel_total : 34610 time to create 1 rle with old method : 0.039872169494628906 length of segment : 301 time for calcul the mask position with numpy : 0.0003001689910888672 nb_pixel_total : 7284 time to create 1 rle with old method : 0.008621454238891602 length of segment : 93 time for calcul the mask position with numpy : 0.0005199909210205078 nb_pixel_total : 16683 time to create 1 rle with old method : 0.020678281784057617 length of segment : 115 time for calcul the mask position with numpy : 0.002997159957885742 nb_pixel_total : 103285 time to create 1 rle with old method : 0.1351785659790039 length of segment : 510 time for calcul the mask position with numpy : 0.0002167224884033203 nb_pixel_total : 5669 time to create 1 rle with old method : 0.00684046745300293 length of segment : 77 time for calcul the mask position with numpy : 0.00015854835510253906 nb_pixel_total : 4937 time to create 1 rle with old method : 0.005898952484130859 length of segment : 128 time for calcul the mask position with numpy : 0.0005631446838378906 nb_pixel_total : 11217 time to create 1 rle with old method : 0.013368844985961914 length of segment : 187 time for calcul the mask position with numpy : 0.0003809928894042969 nb_pixel_total : 8516 time to create 1 rle with old method : 0.010097980499267578 length of segment : 148 time for calcul the mask position with numpy : 0.0004265308380126953 nb_pixel_total : 15015 time to create 1 rle with old method : 0.017634153366088867 length of segment : 112 time for calcul the mask position with numpy : 0.0003528594970703125 nb_pixel_total : 6559 time to create 1 rle with old method : 0.007874011993408203 length of segment : 86 time for calcul the mask position with numpy : 0.0004229545593261719 nb_pixel_total : 8425 time to create 1 rle with old method : 0.010085821151733398 length of segment : 137 time for calcul the mask position with numpy : 0.0002834796905517578 nb_pixel_total : 5885 time to create 1 rle with old method : 0.007123231887817383 length of segment : 90 time for calcul the mask position with numpy : 0.01762866973876953 nb_pixel_total : 737726 time to create 1 rle with new method : 0.02378392219543457 length of segment : 957 time for calcul the mask position with numpy : 0.00025153160095214844 nb_pixel_total : 6060 time to create 1 rle with old method : 0.006955862045288086 length of segment : 83 time for calcul the mask position with numpy : 0.012634754180908203 nb_pixel_total : 776713 time to create 1 rle with new method : 0.12636280059814453 length of segment : 987 time for calcul the mask position with numpy : 0.0005023479461669922 nb_pixel_total : 10941 time to create 1 rle with old method : 0.012959480285644531 length of segment : 124 time for calcul the mask position with numpy : 0.00024890899658203125 nb_pixel_total : 4638 time to create 1 rle with old method : 0.0056231021881103516 length of segment : 88 time spent for convertir_results : 3.0262842178344727 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 35 chid ids of type : 3594 Number RLEs to save : 10881 save missing photos in datou_result : time spend for datou_step_exec : 31.77522921562195 time spend to save output : 1.0439391136169434 total time spend for step 1 : 32.81916832923889 step2:crop_condition Wed Aug 20 08:21:03 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 23 ! batch 1 Loaded 35 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 10 About to insert : list_path_to_insert length 10 new photo from crops ! About to upload 10 photos upload in portfolio : 3736932 init cache_photo without model_param we have 10 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1755670865_1264403 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 10 photos in the portfolio 3736932 time of upload the photos Elapsed time : 2.4428577423095703 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 3736932 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1755670868_1264403 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 5 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.3152930736541748 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3736932 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1755670870_1264403 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 2 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.0492949485778809 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 16 About to insert : list_path_to_insert length 16 new photo from crops ! About to upload 16 photos upload in portfolio : 3736932 init cache_photo without model_param we have 16 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1755670894_1264403 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 16 photos in the portfolio 3736932 time of upload the photos Elapsed time : 3.9417660236358643 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3736932 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1755670899_1264403 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 2 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.8584249019622803 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1377978543, 1377978541, 1377978539, 1377978536, 1377978486, 1377978467, 1377978458, 1377978456, 1377978455, 1377978453, 1377978439, 1377978438, 1377978437, 1377978414, 1377978392, 1377978367, 1377978306, 1377978298, 1377978294, 1377978278, 1377978254, 1377978253, 1377978201] Looping around the photos to save general results len do output : 35 /1377984778Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984779Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984780Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984781Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984782Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984783Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984784Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984785Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984786Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984787Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984791Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984792Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984793Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984794Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984795Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984797Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984798Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984930Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984931Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984932Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984933Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984934Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984935Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984936Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984937Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984938Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984939Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984940Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984941Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984942Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984943Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984944Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984945Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984949Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1377984951Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978543', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978541', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978539', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978536', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978486', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978467', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978458', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978456', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978455', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978453', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978439', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978438', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978437', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978414', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978392', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978367', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978306', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978298', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978294', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978278', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978254', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978253', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978201', None, None, None, None, None, '3555385') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 128 time used for this insertion : 0.09821915626525879 save_final save missing photos in datou_result : time spend for datou_step_exec : 36.17728328704834 time spend to save output : 0.09982800483703613 total time spend for step 2 : 36.277111291885376 step3:rle_unique_nms_with_priority Wed Aug 20 08:21:40 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 35 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.4685649871826172 time for calcul the mask position with numpy : 0.3155944347381592 nb_pixel_total : 2069220 time to create 1 rle with new method : 0.07631540298461914 time for calcul the mask position with numpy : 0.0061492919921875 nb_pixel_total : 4380 time to create 1 rle with old method : 0.005053043365478516 create new chi : 0.41382718086242676 time to delete rle : 0.0211641788482666 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 1252 TO DO : save crop sub photo not yet done ! save time : 0.19831299781799316 No data in photo_id : 1377978541 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.5387880802154541 time for calcul the mask position with numpy : 0.07805395126342773 nb_pixel_total : 1296597 time to create 1 rle with new method : 0.07647538185119629 time for calcul the mask position with numpy : 0.012372493743896484 nb_pixel_total : 765380 time to create 1 rle with new method : 0.08457207679748535 time for calcul the mask position with numpy : 0.007005453109741211 nb_pixel_total : 11623 time to create 1 rle with old method : 0.014661312103271484 create new chi : 0.28313136100769043 time to delete rle : 0.0004956722259521484 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 3310 TO DO : save crop sub photo not yet done ! save time : 0.24549221992492676 No data in photo_id : 1377978536 nb_obj : 3 nb_hashtags : 1 time to prepare the origin masks : 0.10658073425292969 time for calcul the mask position with numpy : 0.034683942794799805 nb_pixel_total : 1160396 time to create 1 rle with new method : 0.3915882110595703 time for calcul the mask position with numpy : 0.006199359893798828 nb_pixel_total : 7651 time to create 1 rle with old method : 0.009413003921508789 time for calcul the mask position with numpy : 0.012079954147338867 nb_pixel_total : 791762 time to create 1 rle with new method : 0.08475089073181152 time for calcul the mask position with numpy : 0.007080078125 nb_pixel_total : 113791 time to create 1 rle with old method : 0.13664531707763672 create new chi : 0.6915955543518066 time to delete rle : 0.0007483959197998047 batch 1 Loaded 7 chid ids of type : 3594 +++Number RLEs to save : 4526 TO DO : save crop sub photo not yet done ! save time : 0.4970581531524658 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.032262563705444336 time for calcul the mask position with numpy : 0.01926565170288086 nb_pixel_total : 2071352 time to create 1 rle with new method : 0.05215334892272949 time for calcul the mask position with numpy : 0.006095170974731445 nb_pixel_total : 2248 time to create 1 rle with old method : 0.0025801658630371094 create new chi : 0.08720064163208008 time to delete rle : 0.00022149085998535156 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 1224 TO DO : save crop sub photo not yet done ! save time : 0.15909552574157715 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.2378525733947754 time for calcul the mask position with numpy : 0.013397216796875 nb_pixel_total : 604597 time to create 1 rle with new method : 0.07415103912353516 time for calcul the mask position with numpy : 0.23916268348693848 nb_pixel_total : 1439652 time to create 1 rle with new method : 0.07074856758117676 time for calcul the mask position with numpy : 0.00626373291015625 nb_pixel_total : 17841 time to create 1 rle with old method : 0.020101070404052734 time for calcul the mask position with numpy : 0.006084918975830078 nb_pixel_total : 11510 time to create 1 rle with old method : 0.012954473495483398 create new chi : 0.45829010009765625 time to delete rle : 0.0005626678466796875 batch 1 Loaded 7 chid ids of type : 3594 +++++++Number RLEs to save : 4305 TO DO : save crop sub photo not yet done ! save time : 0.3127632141113281 No data in photo_id : 1377978456 No data in photo_id : 1377978455 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.0506901741027832 time for calcul the mask position with numpy : 0.10291004180908203 nb_pixel_total : 1330939 time to create 1 rle with new method : 0.08152461051940918 time for calcul the mask position with numpy : 0.011111736297607422 nb_pixel_total : 720260 time to create 1 rle with new method : 0.2779355049133301 time for calcul the mask position with numpy : 0.007410526275634766 nb_pixel_total : 2690 time to create 1 rle with old method : 0.0030121803283691406 time for calcul the mask position with numpy : 0.006337165832519531 nb_pixel_total : 19711 time to create 1 rle with old method : 0.02198195457458496 create new chi : 0.5220215320587158 time to delete rle : 0.0005803108215332031 batch 1 Loaded 7 chid ids of type : 3594 ++++Number RLEs to save : 3646 TO DO : save crop sub photo not yet done ! save time : 0.35463523864746094 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.08850836753845215 time for calcul the mask position with numpy : 0.03375077247619629 nb_pixel_total : 2062937 time to create 1 rle with new method : 0.033260345458984375 time for calcul the mask position with numpy : 0.00697636604309082 nb_pixel_total : 10663 time to create 1 rle with old method : 0.012018918991088867 create new chi : 0.09360718727111816 time to delete rle : 0.0003132820129394531 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 1450 TO DO : save crop sub photo not yet done ! save time : 0.18182706832885742 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.16156220436096191 time for calcul the mask position with numpy : 0.18241333961486816 nb_pixel_total : 2053256 time to create 1 rle with new method : 0.2059180736541748 time for calcul the mask position with numpy : 0.0067865848541259766 nb_pixel_total : 8082 time to create 1 rle with old method : 0.009088993072509766 time for calcul the mask position with numpy : 0.006039142608642578 nb_pixel_total : 12262 time to create 1 rle with old method : 0.013686180114746094 create new chi : 0.43340182304382324 time to delete rle : 0.0002770423889160156 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 1620 TO DO : save crop sub photo not yet done ! save time : 0.16144633293151855 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.039017438888549805 time for calcul the mask position with numpy : 0.05755186080932617 nb_pixel_total : 2051958 time to create 1 rle with new method : 0.1971759796142578 time for calcul the mask position with numpy : 0.006300687789916992 nb_pixel_total : 19387 time to create 1 rle with old method : 0.021850109100341797 time for calcul the mask position with numpy : 0.006031513214111328 nb_pixel_total : 2255 time to create 1 rle with old method : 0.0025691986083984375 create new chi : 0.3012051582336426 time to delete rle : 0.0002791881561279297 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 1618 TO DO : save crop sub photo not yet done ! save time : 0.21065425872802734 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.03271889686584473 time for calcul the mask position with numpy : 0.03855752944946289 nb_pixel_total : 2038990 time to create 1 rle with new method : 0.14680147171020508 time for calcul the mask position with numpy : 0.006381034851074219 nb_pixel_total : 34610 time to create 1 rle with old method : 0.03886246681213379 create new chi : 0.23913908004760742 time to delete rle : 0.00025582313537597656 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 1682 TO DO : save crop sub photo not yet done ! save time : 0.18114471435546875 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.0322723388671875 time for calcul the mask position with numpy : 0.0381474494934082 nb_pixel_total : 2066316 time to create 1 rle with new method : 0.0948176383972168 time for calcul the mask position with numpy : 0.0064411163330078125 nb_pixel_total : 7284 time to create 1 rle with old method : 0.008268594741821289 create new chi : 0.1586301326751709 time to delete rle : 0.00026106834411621094 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 1266 TO DO : save crop sub photo not yet done ! save time : 0.15003395080566406 nb_obj : 2 nb_hashtags : 1 time to prepare the origin masks : 0.06167244911193848 time for calcul the mask position with numpy : 0.17902159690856934 nb_pixel_total : 1953632 time to create 1 rle with new method : 0.07819414138793945 time for calcul the mask position with numpy : 0.007352113723754883 nb_pixel_total : 103285 time to create 1 rle with old method : 0.11488842964172363 time for calcul the mask position with numpy : 0.006478309631347656 nb_pixel_total : 16683 time to create 1 rle with old method : 0.01871657371520996 create new chi : 0.41599559783935547 time to delete rle : 0.0003802776336669922 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 2330 TO DO : save crop sub photo not yet done ! save time : 0.28124046325683594 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.20255398750305176 time for calcul the mask position with numpy : 0.05834245681762695 nb_pixel_total : 2062994 time to create 1 rle with new method : 0.18843889236450195 time for calcul the mask position with numpy : 0.006186485290527344 nb_pixel_total : 4937 time to create 1 rle with old method : 0.0053293704986572266 time for calcul the mask position with numpy : 0.006540775299072266 nb_pixel_total : 5669 time to create 1 rle with old method : 0.0061223506927490234 create new chi : 0.28093457221984863 time to delete rle : 0.0003025531768798828 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 1490 TO DO : save crop sub photo not yet done ! save time : 0.23266386985778809 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.03798222541809082 time for calcul the mask position with numpy : 0.08122777938842773 nb_pixel_total : 2062383 time to create 1 rle with new method : 0.16501474380493164 time for calcul the mask position with numpy : 0.0062906742095947266 nb_pixel_total : 11217 time to create 1 rle with old method : 0.012623071670532227 create new chi : 0.2737307548522949 time to delete rle : 0.00025272369384765625 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 1454 TO DO : save crop sub photo not yet done ! save time : 0.1982252597808838 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.03313040733337402 time for calcul the mask position with numpy : 0.039310455322265625 nb_pixel_total : 2065084 time to create 1 rle with new method : 0.08638739585876465 time for calcul the mask position with numpy : 0.006219387054443359 nb_pixel_total : 8516 time to create 1 rle with old method : 0.009636402130126953 create new chi : 0.15222454071044922 time to delete rle : 0.0002434253692626953 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 1376 TO DO : save crop sub photo not yet done ! save time : 0.12726378440856934 nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 0.28486108779907227 time for calcul the mask position with numpy : 0.1484236717224121 nb_pixel_total : 2037716 time to create 1 rle with new method : 0.19530892372131348 time for calcul the mask position with numpy : 0.006442070007324219 nb_pixel_total : 5885 time to create 1 rle with old method : 0.006736040115356445 time for calcul the mask position with numpy : 0.006386756896972656 nb_pixel_total : 8425 time to create 1 rle with old method : 0.009485006332397461 time for calcul the mask position with numpy : 0.006382465362548828 nb_pixel_total : 6559 time to create 1 rle with old method : 0.007391929626464844 time for calcul the mask position with numpy : 0.006304740905761719 nb_pixel_total : 15015 time to create 1 rle with old method : 0.01686263084411621 create new chi : 0.4194681644439697 time to delete rle : 0.0003819465637207031 batch 1 Loaded 9 chid ids of type : 3594 ++++Number RLEs to save : 1930 TO DO : save crop sub photo not yet done ! save time : 0.24868130683898926 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.04284381866455078 time for calcul the mask position with numpy : 0.015751361846923828 nb_pixel_total : 1335874 time to create 1 rle with new method : 0.08080720901489258 time for calcul the mask position with numpy : 0.014878273010253906 nb_pixel_total : 737726 time to create 1 rle with new method : 0.03037238121032715 create new chi : 0.14223670959472656 time to delete rle : 0.0002994537353515625 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 2994 TO DO : save crop sub photo not yet done ! save time : 0.25415658950805664 nb_obj : 2 nb_hashtags : 1 time to prepare the origin masks : 0.051558732986450195 time for calcul the mask position with numpy : 0.10614871978759766 nb_pixel_total : 1290827 time to create 1 rle with new method : 0.07176518440246582 time for calcul the mask position with numpy : 0.011832237243652344 nb_pixel_total : 776713 time to create 1 rle with new method : 0.12555980682373047 time for calcul the mask position with numpy : 0.006419658660888672 nb_pixel_total : 6060 time to create 1 rle with old method : 0.0068318843841552734 create new chi : 0.336437463760376 time to delete rle : 0.000598907470703125 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 3220 TO DO : save crop sub photo not yet done ! save time : 0.24334263801574707 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.17656970024108887 time for calcul the mask position with numpy : 0.08507061004638672 nb_pixel_total : 2058021 time to create 1 rle with new method : 0.14885997772216797 time for calcul the mask position with numpy : 0.006319761276245117 nb_pixel_total : 4638 time to create 1 rle with old method : 0.00536656379699707 time for calcul the mask position with numpy : 0.0069043636322021484 nb_pixel_total : 10941 time to create 1 rle with old method : 0.01231074333190918 create new chi : 0.27627992630004883 time to delete rle : 0.0002551078796386719 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 1504 TO DO : save crop sub photo not yet done ! save time : 0.1773231029510498 map_output_result : {1377978543: (0.0, 'Should be the crop_list due to order', 0), 1377978541: (0.0, 'Should be the crop_list due to order', 0.0), 1377978539: (0.0, 'Should be the crop_list due to order', 0), 1377978536: (0.0, 'Should be the crop_list due to order', 0.0), 1377978486: (0.0, 'Should be the crop_list due to order', 0), 1377978467: (0.0, 'Should be the crop_list due to order', 0), 1377978458: (0.0, 'Should be the crop_list due to order', 0), 1377978456: (0.0, 'Should be the crop_list due to order', 0.0), 1377978455: (0.0, 'Should be the crop_list due to order', 0.0), 1377978453: (0.0, 'Should be the crop_list due to order', 0), 1377978439: (0.0, 'Should be the crop_list due to order', 0), 1377978438: (0.0, 'Should be the crop_list due to order', 0), 1377978437: (0.0, 'Should be the crop_list due to order', 0), 1377978414: (0.0, 'Should be the crop_list due to order', 0), 1377978392: (0.0, 'Should be the crop_list due to order', 0), 1377978367: (0.0, 'Should be the crop_list due to order', 0), 1377978306: (0.0, 'Should be the crop_list due to order', 0), 1377978298: (0.0, 'Should be the crop_list due to order', 0), 1377978294: (0.0, 'Should be the crop_list due to order', 0), 1377978278: (0.0, 'Should be the crop_list due to order', 0), 1377978254: (0.0, 'Should be the crop_list due to order', 0), 1377978253: (0.0, 'Should be the crop_list due to order', 0), 1377978201: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1377978543, 1377978541, 1377978539, 1377978536, 1377978486, 1377978467, 1377978458, 1377978456, 1377978455, 1377978453, 1377978439, 1377978438, 1377978437, 1377978414, 1377978392, 1377978367, 1377978306, 1377978298, 1377978294, 1377978278, 1377978254, 1377978253, 1377978201] Looping around the photos to save general results len do output : 23 /1377978543.Didn't retrieve data . /1377978541.Didn't retrieve data . /1377978539.Didn't retrieve data . /1377978536.Didn't retrieve data . /1377978486.Didn't retrieve data . /1377978467.Didn't retrieve data . /1377978458.Didn't retrieve data . /1377978456.Didn't retrieve data . /1377978455.Didn't retrieve data . /1377978453.Didn't retrieve data . /1377978439.Didn't retrieve data . /1377978438.Didn't retrieve data . /1377978437.Didn't retrieve data . /1377978414.Didn't retrieve data . /1377978392.Didn't retrieve data . /1377978367.Didn't retrieve data . /1377978306.Didn't retrieve data . /1377978298.Didn't retrieve data . /1377978294.Didn't retrieve data . /1377978278.Didn't retrieve data . /1377978254.Didn't retrieve data . /1377978253.Didn't retrieve data . /1377978201.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978543', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978541', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978539', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978536', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978486', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978467', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978458', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978456', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978455', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978453', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978439', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978438', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978437', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978414', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978392', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978367', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978306', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978298', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978294', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978278', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978254', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978253', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978201', None, None, None, None, None, '3555385') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 69 time used for this insertion : 0.0665736198425293 save_final save missing photos in datou_result : time spend for datou_step_exec : 15.016069650650024 time spend to save output : 0.06743168830871582 total time spend for step 3 : 15.08350133895874 step4:ventilate_hashtags_in_portfolio Wed Aug 20 08:21:55 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 26070739 get user id for portfolio 26070739 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26070739 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('autre','mal_croppe','background','carton','environnement','pet_clair','papier','pehd','flou','pet_fonce','metal')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26070739 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('autre','mal_croppe','background','carton','environnement','pet_clair','papier','pehd','flou','pet_fonce','metal')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26070739 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('autre','mal_croppe','background','carton','environnement','pet_clair','papier','pehd','flou','pet_fonce','metal')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/26071060,26071061,26071062,26071063,26071064,26071065,26071066,26071067,26071068,26071069,26071070?tags=autre,mal_croppe,background,carton,environnement,pet_clair,papier,pehd,flou,pet_fonce,metal Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1377978543, 1377978541, 1377978539, 1377978536, 1377978486, 1377978467, 1377978458, 1377978456, 1377978455, 1377978453, 1377978439, 1377978438, 1377978437, 1377978414, 1377978392, 1377978367, 1377978306, 1377978298, 1377978294, 1377978278, 1377978254, 1377978253, 1377978201] Looping around the photos to save general results len do output : 1 /26070739. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978543', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978541', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978539', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978536', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978486', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978467', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978458', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978456', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978455', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978453', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978439', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978438', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978437', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978414', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978392', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978367', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978306', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978298', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978294', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978278', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978254', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978253', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978201', None, None, None, None, None, '3555385') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 24 time used for this insertion : 0.029030561447143555 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.8802762031555176 time spend to save output : 0.029365062713623047 total time spend for step 4 : 1.9096412658691406 step5:final Wed Aug 20 08:21:57 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1377978543: ('0.12004560436795493',), 1377978541: ('0.12004560436795493',), 1377978539: ('0.12004560436795493',), 1377978536: ('0.12004560436795493',), 1377978486: ('0.12004560436795493',), 1377978467: ('0.12004560436795493',), 1377978458: ('0.12004560436795493',), 1377978456: ('0.12004560436795493',), 1377978455: ('0.12004560436795493',), 1377978453: ('0.12004560436795493',), 1377978439: ('0.12004560436795493',), 1377978438: ('0.12004560436795493',), 1377978437: ('0.12004560436795493',), 1377978414: ('0.12004560436795493',), 1377978392: ('0.12004560436795493',), 1377978367: ('0.12004560436795493',), 1377978306: ('0.12004560436795493',), 1377978298: ('0.12004560436795493',), 1377978294: ('0.12004560436795493',), 1377978278: ('0.12004560436795493',), 1377978254: ('0.12004560436795493',), 1377978253: ('0.12004560436795493',), 1377978201: ('0.12004560436795493',)} new output for save of step final : {1377978543: ('0.12004560436795493',), 1377978541: ('0.12004560436795493',), 1377978539: ('0.12004560436795493',), 1377978536: ('0.12004560436795493',), 1377978486: ('0.12004560436795493',), 1377978467: ('0.12004560436795493',), 1377978458: ('0.12004560436795493',), 1377978456: ('0.12004560436795493',), 1377978455: ('0.12004560436795493',), 1377978453: ('0.12004560436795493',), 1377978439: ('0.12004560436795493',), 1377978438: ('0.12004560436795493',), 1377978437: ('0.12004560436795493',), 1377978414: ('0.12004560436795493',), 1377978392: ('0.12004560436795493',), 1377978367: ('0.12004560436795493',), 1377978306: ('0.12004560436795493',), 1377978298: ('0.12004560436795493',), 1377978294: ('0.12004560436795493',), 1377978278: ('0.12004560436795493',), 1377978254: ('0.12004560436795493',), 1377978253: ('0.12004560436795493',), 1377978201: ('0.12004560436795493',)} [1377978543, 1377978541, 1377978539, 1377978536, 1377978486, 1377978467, 1377978458, 1377978456, 1377978455, 1377978453, 1377978439, 1377978438, 1377978437, 1377978414, 1377978392, 1377978367, 1377978306, 1377978298, 1377978294, 1377978278, 1377978254, 1377978253, 1377978201] Looping around the photos to save general results len do output : 23 /1377978543.Didn't retrieve data . /1377978541.Didn't retrieve data . /1377978539.Didn't retrieve data . /1377978536.Didn't retrieve data . /1377978486.Didn't retrieve data . /1377978467.Didn't retrieve data . /1377978458.Didn't retrieve data . /1377978456.Didn't retrieve data . /1377978455.Didn't retrieve data . /1377978453.Didn't retrieve data . /1377978439.Didn't retrieve data . /1377978438.Didn't retrieve data . /1377978437.Didn't retrieve data . /1377978414.Didn't retrieve data . /1377978392.Didn't retrieve data . /1377978367.Didn't retrieve data . /1377978306.Didn't retrieve data . /1377978298.Didn't retrieve data . /1377978294.Didn't retrieve data . /1377978278.Didn't retrieve data . /1377978254.Didn't retrieve data . /1377978253.Didn't retrieve data . /1377978201.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978543', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978541', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978539', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978536', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978486', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978467', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978458', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978456', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978455', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978453', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978439', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978438', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978437', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978414', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978392', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978367', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978306', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978298', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978294', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978278', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978254', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978253', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978201', None, None, None, None, None, '3555385') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 69 time used for this insertion : 0.027489423751831055 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.13814997673034668 time spend to save output : 0.028528451919555664 total time spend for step 5 : 0.16667842864990234 step6:blur_detection Wed Aug 20 08:21:57 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1755670828_1264403_1377978543_bef6082a47740d138c6e5a8a71911c8c.jpg resize: (1080, 1920) 1377978543 -2.6379617289136164 treat image : temp/1755670828_1264403_1377978541_8e8e18072b7910283df608f62babd629.jpg resize: (1080, 1920) 1377978541 -2.7814023087590725 treat image : temp/1755670828_1264403_1377978539_2e640831a487c9d26853c2365b84236a.jpg resize: (1080, 1920) 1377978539 0.9355133604760646 treat image : temp/1755670828_1264403_1377978536_4308190ec46d5c54b4f23a47c23ed9bb.jpg resize: (1080, 1920) 1377978536 -3.145618438348134 treat image : temp/1755670828_1264403_1377978486_a68a191b07e67c76985531250a615706.jpg resize: (1080, 1920) 1377978486 -2.2355993154604668 treat image : temp/1755670828_1264403_1377978467_deb3a616c9d675b6da6c3b8a431e19fe.jpg resize: (1080, 1920) 1377978467 -2.6287363879616388 treat image : temp/1755670828_1264403_1377978458_f8647888a2356ebdea2aaddb8343e5e4.jpg resize: (1080, 1920) 1377978458 -0.7556555169577319 treat image : temp/1755670828_1264403_1377978456_0535ae4036913ac9723be58ae0ce49c8.jpg resize: (1080, 1920) 1377978456 -1.5461356854033614 treat image : temp/1755670828_1264403_1377978455_2d1618d57a81992c8bc00369da19ae14.jpg resize: (1080, 1920) 1377978455 -2.997019578213948 treat image : temp/1755670828_1264403_1377978453_f24ff9d85d6d23a838c384cea9e5801d.jpg resize: (1080, 1920) 1377978453 -2.6468294007803936 treat image : temp/1755670828_1264403_1377978439_a4db036d8e85e388b7b5403d3b20708c.jpg resize: (1080, 1920) 1377978439 -1.992817526285214 treat image : temp/1755670828_1264403_1377978438_d6392b623796f59d152cc6e787803230.jpg resize: (1080, 1920) 1377978438 -2.5077092143196276 treat image : temp/1755670828_1264403_1377978437_90a13540ff47a6db4220f2253cbbc3e6.jpg resize: (1080, 1920) 1377978437 -2.5730546744657787 treat image : temp/1755670828_1264403_1377978414_ae74105ff53ce301807c73516205cb53.jpg resize: (1080, 1920) 1377978414 -3.2612411075490337 treat image : temp/1755670828_1264403_1377978392_385d4f77c5467f8c2c96eb2a01145845.jpg resize: (1080, 1920) 1377978392 -2.4501391222103264 treat image : temp/1755670828_1264403_1377978367_43fc77aeba85445e9931bb02735f369f.jpg resize: (1080, 1920) 1377978367 -1.740718560442027 treat image : temp/1755670828_1264403_1377978306_42daf3e15473929f1d8da2d29f73cc04.jpg resize: (1080, 1920) 1377978306 -1.8121098176009531 treat image : temp/1755670828_1264403_1377978298_fc00b091a0d6fb24c75536f8f6e25096.jpg resize: (1080, 1920) 1377978298 -0.375401293333239 treat image : temp/1755670828_1264403_1377978294_29c4b3208b10b19ce639e8ceab333d62.jpg resize: (1080, 1920) 1377978294 -2.9054487599442713 treat image : temp/1755670828_1264403_1377978278_6ae5e12f0bd78e9c5ddac4ac4b43d325.jpg resize: (1080, 1920) 1377978278 -2.1157578577321585 treat image : temp/1755670828_1264403_1377978254_54778f17067bbcc3ded23252be97efbb.jpg resize: (1080, 1920) 1377978254 0.9056591519737623 treat image : temp/1755670828_1264403_1377978253_e3aa5bc9c7af3f49d26e26d8909171a9.jpg resize: (1080, 1920) 1377978253 -1.4129297440780777 treat image : temp/1755670828_1264403_1377978201_7c194563359ba87282d64fdf6b214d36.jpg resize: (1080, 1920) 1377978201 -2.292379392122436 treat image : temp/1755670828_1264403_1377978539_2e640831a487c9d26853c2365b84236a_rle_crop_3921286132_0.png resize: (117, 135) 1377984778 -1.1564144058967472 treat image : temp/1755670828_1264403_1377978467_deb3a616c9d675b6da6c3b8a431e19fe_rle_crop_3921286137_0.png resize: (72, 54) 1377984779 -2.2994334369599483 treat image : temp/1755670828_1264403_1377978458_f8647888a2356ebdea2aaddb8343e5e4_rle_crop_3921286138_0.png resize: (154, 118) 1377984780 -1.4269026060478385 treat image : temp/1755670828_1264403_1377978439_a4db036d8e85e388b7b5403d3b20708c_rle_crop_3921286144_0.png resize: (184, 112) 1377984781 -1.4439995684005207 treat image : temp/1755670828_1264403_1377978438_d6392b623796f59d152cc6e787803230_rle_crop_3921286145_0.png resize: (153, 135) 1377984782 -1.5932973979065144 treat image : temp/1755670828_1264403_1377978437_90a13540ff47a6db4220f2253cbbc3e6_rle_crop_3921286148_0.png resize: (160, 198) 1377984783 -1.983600917044295 treat image : temp/1755670828_1264403_1377978392_385d4f77c5467f8c2c96eb2a01145845_rle_crop_3921286150_0.png resize: (92, 112) 1377984784 -1.22565852924614 treat image : temp/1755670828_1264403_1377978306_42daf3e15473929f1d8da2d29f73cc04_rle_crop_3921286153_0.png resize: (77, 113) 1377984785 -0.9113132167039174 treat image : temp/1755670828_1264403_1377978298_fc00b091a0d6fb24c75536f8f6e25096_rle_crop_3921286155_0.png resize: (187, 121) 1377984786 -1.3856278189040678 treat image : temp/1755670828_1264403_1377978201_7c194563359ba87282d64fdf6b214d36_rle_crop_3921286165_0.png resize: (88, 102) 1377984787 -1.1719009711728132 treat image : temp/1755670828_1264403_1377978543_bef6082a47740d138c6e5a8a71911c8c_rle_crop_3921286131_0.png resize: (86, 76) 1377984791 -0.794061771942003 treat image : temp/1755670828_1264403_1377978437_90a13540ff47a6db4220f2253cbbc3e6_rle_crop_3921286147_0.png resize: (65, 63) 1377984792 -2.1384984674822203 treat image : temp/1755670828_1264403_1377978294_29c4b3208b10b19ce639e8ceab333d62_rle_crop_3921286156_0.png resize: (148, 96) 1377984793 -1.9809517026383598 treat image : temp/1755670828_1264403_1377978278_6ae5e12f0bd78e9c5ddac4ac4b43d325_rle_crop_3921286158_0.png resize: (86, 104) 1377984794 0.041745141165438836 treat image : temp/1755670828_1264403_1377978201_7c194563359ba87282d64fdf6b214d36_rle_crop_3921286164_0.png resize: (124, 102) 1377984795 1.7190628184732617 treat image : temp/1755670828_1264403_1377978453_f24ff9d85d6d23a838c384cea9e5801d_rle_crop_3921286142_0.png resize: (45, 89) 1377984797 -3.2824319211356494 treat image : temp/1755670828_1264403_1377978438_d6392b623796f59d152cc6e787803230_rle_crop_3921286146_0.png resize: (117, 94) 1377984798 -0.9746340065855557 treat image : temp/1755670828_1264403_1377978539_2e640831a487c9d26853c2365b84236a_rle_crop_3921286133_0.png resize: (993, 1070) 1377984930 0.5716702061425568 treat image : temp/1755670828_1264403_1377978486_a68a191b07e67c76985531250a615706_rle_crop_3921286135_0.png resize: (987, 1143) 1377984931 -0.30408922765976254 treat image : temp/1755670828_1264403_1377978486_a68a191b07e67c76985531250a615706_rle_crop_3921286136_0.png resize: (117, 83) 1377984932 -1.07084999478807 treat image : temp/1755670828_1264403_1377978486_a68a191b07e67c76985531250a615706_rle_crop_3921286134_0.png resize: (605, 314) 1377984933 -0.2819407608352308 treat image : temp/1755670828_1264403_1377978458_f8647888a2356ebdea2aaddb8343e5e4_rle_crop_3921286140_0.png resize: (982, 1649) 1377984934 -0.9798448500962876 treat image : temp/1755670828_1264403_1377978458_f8647888a2356ebdea2aaddb8343e5e4_rle_crop_3921286139_0.png resize: (187, 167) 1377984935 -4.098022798841839 treat image : temp/1755670828_1264403_1377978453_f24ff9d85d6d23a838c384cea9e5801d_rle_crop_3921286143_0.png resize: (927, 980) 1377984936 0.09750361082571862 treat image : temp/1755670828_1264403_1377978453_f24ff9d85d6d23a838c384cea9e5801d_rle_crop_3921286141_0.png resize: (234, 164) 1377984937 -3.215890096737532 treat image : temp/1755670828_1264403_1377978367_43fc77aeba85445e9931bb02735f369f_rle_crop_3921286151_0.png resize: (115, 178) 1377984938 -3.649537889274602 treat image : temp/1755670828_1264403_1377978367_43fc77aeba85445e9931bb02735f369f_rle_crop_3921286152_0.png resize: (491, 355) 1377984939 -0.4375217713925642 treat image : temp/1755670828_1264403_1377978278_6ae5e12f0bd78e9c5ddac4ac4b43d325_rle_crop_3921286160_0.png resize: (89, 80) 1377984940 -1.9612296088948327 treat image : temp/1755670828_1264403_1377978278_6ae5e12f0bd78e9c5ddac4ac4b43d325_rle_crop_3921286157_0.png resize: (112, 169) 1377984941 -3.1749999207665196 treat image : temp/1755670828_1264403_1377978278_6ae5e12f0bd78e9c5ddac4ac4b43d325_rle_crop_3921286159_0.png resize: (110, 105) 1377984942 -1.4968855227905415 treat image : temp/1755670828_1264403_1377978254_54778f17067bbcc3ded23252be97efbb_rle_crop_3921286161_0.png resize: (941, 1024) 1377984943 1.0447524451646233 treat image : temp/1755670828_1264403_1377978253_e3aa5bc9c7af3f49d26e26d8909171a9_rle_crop_3921286163_0.png resize: (983, 1133) 1377984944 -0.009523939009133883 treat image : temp/1755670828_1264403_1377978253_e3aa5bc9c7af3f49d26e26d8909171a9_rle_crop_3921286162_0.png resize: (83, 86) 1377984945 -2.284677130814247 treat image : temp/1755670828_1264403_1377978414_ae74105ff53ce301807c73516205cb53_rle_crop_3921286149_0.png resize: (278, 159) 1377984949 -0.6723159579248243 treat image : temp/1755670828_1264403_1377978306_42daf3e15473929f1d8da2d29f73cc04_rle_crop_3921286154_0.png resize: (128, 50) 1377984951 0.573892868155069 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 58 time used for this insertion : 0.015315532684326172 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 58 time used for this insertion : 0.33898496627807617 save missing photos in datou_result : time spend for datou_step_exec : 21.473289251327515 time spend to save output : 0.3597867488861084 total time spend for step 6 : 21.833076000213623 step7:brightness Wed Aug 20 08:22:19 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1755670828_1264403_1377978543_bef6082a47740d138c6e5a8a71911c8c.jpg treat image : temp/1755670828_1264403_1377978541_8e8e18072b7910283df608f62babd629.jpg treat image : temp/1755670828_1264403_1377978539_2e640831a487c9d26853c2365b84236a.jpg treat image : temp/1755670828_1264403_1377978536_4308190ec46d5c54b4f23a47c23ed9bb.jpg treat image : temp/1755670828_1264403_1377978486_a68a191b07e67c76985531250a615706.jpg treat image : temp/1755670828_1264403_1377978467_deb3a616c9d675b6da6c3b8a431e19fe.jpg treat image : temp/1755670828_1264403_1377978458_f8647888a2356ebdea2aaddb8343e5e4.jpg treat image : temp/1755670828_1264403_1377978456_0535ae4036913ac9723be58ae0ce49c8.jpg treat image : temp/1755670828_1264403_1377978455_2d1618d57a81992c8bc00369da19ae14.jpg treat image : temp/1755670828_1264403_1377978453_f24ff9d85d6d23a838c384cea9e5801d.jpg treat image : temp/1755670828_1264403_1377978439_a4db036d8e85e388b7b5403d3b20708c.jpg treat image : temp/1755670828_1264403_1377978438_d6392b623796f59d152cc6e787803230.jpg treat image : temp/1755670828_1264403_1377978437_90a13540ff47a6db4220f2253cbbc3e6.jpg treat image : temp/1755670828_1264403_1377978414_ae74105ff53ce301807c73516205cb53.jpg treat image : temp/1755670828_1264403_1377978392_385d4f77c5467f8c2c96eb2a01145845.jpg treat image : temp/1755670828_1264403_1377978367_43fc77aeba85445e9931bb02735f369f.jpg treat image : temp/1755670828_1264403_1377978306_42daf3e15473929f1d8da2d29f73cc04.jpg treat image : temp/1755670828_1264403_1377978298_fc00b091a0d6fb24c75536f8f6e25096.jpg treat image : temp/1755670828_1264403_1377978294_29c4b3208b10b19ce639e8ceab333d62.jpg treat image : temp/1755670828_1264403_1377978278_6ae5e12f0bd78e9c5ddac4ac4b43d325.jpg treat image : temp/1755670828_1264403_1377978254_54778f17067bbcc3ded23252be97efbb.jpg treat image : temp/1755670828_1264403_1377978253_e3aa5bc9c7af3f49d26e26d8909171a9.jpg treat image : temp/1755670828_1264403_1377978201_7c194563359ba87282d64fdf6b214d36.jpg treat image : temp/1755670828_1264403_1377978539_2e640831a487c9d26853c2365b84236a_rle_crop_3921286132_0.png treat image : temp/1755670828_1264403_1377978467_deb3a616c9d675b6da6c3b8a431e19fe_rle_crop_3921286137_0.png treat image : temp/1755670828_1264403_1377978458_f8647888a2356ebdea2aaddb8343e5e4_rle_crop_3921286138_0.png treat image : temp/1755670828_1264403_1377978439_a4db036d8e85e388b7b5403d3b20708c_rle_crop_3921286144_0.png treat image : temp/1755670828_1264403_1377978438_d6392b623796f59d152cc6e787803230_rle_crop_3921286145_0.png treat image : temp/1755670828_1264403_1377978437_90a13540ff47a6db4220f2253cbbc3e6_rle_crop_3921286148_0.png treat image : temp/1755670828_1264403_1377978392_385d4f77c5467f8c2c96eb2a01145845_rle_crop_3921286150_0.png treat image : temp/1755670828_1264403_1377978306_42daf3e15473929f1d8da2d29f73cc04_rle_crop_3921286153_0.png treat image : temp/1755670828_1264403_1377978298_fc00b091a0d6fb24c75536f8f6e25096_rle_crop_3921286155_0.png treat image : temp/1755670828_1264403_1377978201_7c194563359ba87282d64fdf6b214d36_rle_crop_3921286165_0.png treat image : temp/1755670828_1264403_1377978543_bef6082a47740d138c6e5a8a71911c8c_rle_crop_3921286131_0.png treat image : temp/1755670828_1264403_1377978437_90a13540ff47a6db4220f2253cbbc3e6_rle_crop_3921286147_0.png treat image : temp/1755670828_1264403_1377978294_29c4b3208b10b19ce639e8ceab333d62_rle_crop_3921286156_0.png treat image : temp/1755670828_1264403_1377978278_6ae5e12f0bd78e9c5ddac4ac4b43d325_rle_crop_3921286158_0.png treat image : temp/1755670828_1264403_1377978201_7c194563359ba87282d64fdf6b214d36_rle_crop_3921286164_0.png treat image : temp/1755670828_1264403_1377978453_f24ff9d85d6d23a838c384cea9e5801d_rle_crop_3921286142_0.png treat image : temp/1755670828_1264403_1377978438_d6392b623796f59d152cc6e787803230_rle_crop_3921286146_0.png treat image : temp/1755670828_1264403_1377978539_2e640831a487c9d26853c2365b84236a_rle_crop_3921286133_0.png treat image : temp/1755670828_1264403_1377978486_a68a191b07e67c76985531250a615706_rle_crop_3921286135_0.png treat image : temp/1755670828_1264403_1377978486_a68a191b07e67c76985531250a615706_rle_crop_3921286136_0.png treat image : temp/1755670828_1264403_1377978486_a68a191b07e67c76985531250a615706_rle_crop_3921286134_0.png treat image : temp/1755670828_1264403_1377978458_f8647888a2356ebdea2aaddb8343e5e4_rle_crop_3921286140_0.png treat image : temp/1755670828_1264403_1377978458_f8647888a2356ebdea2aaddb8343e5e4_rle_crop_3921286139_0.png treat image : temp/1755670828_1264403_1377978453_f24ff9d85d6d23a838c384cea9e5801d_rle_crop_3921286143_0.png treat image : temp/1755670828_1264403_1377978453_f24ff9d85d6d23a838c384cea9e5801d_rle_crop_3921286141_0.png treat image : temp/1755670828_1264403_1377978367_43fc77aeba85445e9931bb02735f369f_rle_crop_3921286151_0.png treat image : temp/1755670828_1264403_1377978367_43fc77aeba85445e9931bb02735f369f_rle_crop_3921286152_0.png treat image : temp/1755670828_1264403_1377978278_6ae5e12f0bd78e9c5ddac4ac4b43d325_rle_crop_3921286160_0.png treat image : temp/1755670828_1264403_1377978278_6ae5e12f0bd78e9c5ddac4ac4b43d325_rle_crop_3921286157_0.png treat image : temp/1755670828_1264403_1377978278_6ae5e12f0bd78e9c5ddac4ac4b43d325_rle_crop_3921286159_0.png treat image : temp/1755670828_1264403_1377978254_54778f17067bbcc3ded23252be97efbb_rle_crop_3921286161_0.png treat image : temp/1755670828_1264403_1377978253_e3aa5bc9c7af3f49d26e26d8909171a9_rle_crop_3921286163_0.png treat image : temp/1755670828_1264403_1377978253_e3aa5bc9c7af3f49d26e26d8909171a9_rle_crop_3921286162_0.png treat image : temp/1755670828_1264403_1377978414_ae74105ff53ce301807c73516205cb53_rle_crop_3921286149_0.png treat image : temp/1755670828_1264403_1377978306_42daf3e15473929f1d8da2d29f73cc04_rle_crop_3921286154_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 58 time used for this insertion : 0.02104473114013672 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 58 time used for this insertion : 0.4682579040527344 save missing photos in datou_result : time spend for datou_step_exec : 5.796828269958496 time spend to save output : 0.4943969249725342 total time spend for step 7 : 6.29122519493103 step8:velours_tree Wed Aug 20 08:22:25 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.32671189308166504 time spend to save output : 9.179115295410156e-05 total time spend for step 8 : 0.32680368423461914 step9:send_mail_cod Wed Aug 20 08:22:25 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P26070739_20-08-2025_08_22_25.pdf 26071060 change filename to text .change filename to text .imagette260710601755670945 26071061 imagette260710611755670945 26071062 imagette260710621755670945 26071063 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette260710631755670945 26071065 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette260710651755670946 26071066 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette260710661755670947 26071067 imagette260710671755670948 26071068 imagette260710681755670948 26071069 imagette260710691755670948 26071070 change filename to text .change filename to text .imagette260710701755670948 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=26070739 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/26071060,26071061,26071062,26071063,26071064,26071065,26071066,26071067,26071068,26071069,26071070?tags=autre,mal_croppe,background,carton,environnement,pet_clair,papier,pehd,flou,pet_fonce,metal args[1377978543] : ((1377978543, -2.6379617289136164, 492609224), (1377978543, 0.5935240302302908, 2107752395), '0.12004560436795493') We are sending mail with results at report@fotonower.com args[1377978541] : ((1377978541, -2.7814023087590725, 492609224), (1377978541, 0.41956417081057723, 2107752395), '0.12004560436795493') We are sending mail with results at report@fotonower.com args[1377978539] : ((1377978539, 0.9355133604760646, 492688767), (1377978539, 0.4476414607873166, 2107752395), '0.12004560436795493') We are sending mail with results at report@fotonower.com args[1377978536] : ((1377978536, -3.145618438348134, 492609224), (1377978536, 0.40611656268405694, 2107752395), '0.12004560436795493') We are sending mail with results at report@fotonower.com args[1377978486] : ((1377978486, -2.2355993154604668, 492609224), (1377978486, 0.6268975244650729, 2107752395), '0.12004560436795493') We are sending mail with results at report@fotonower.com args[1377978467] : ((1377978467, -2.6287363879616388, 492609224), (1377978467, 0.4732598051992247, 2107752395), '0.12004560436795493') We are sending mail with results at report@fotonower.com args[1377978458] : ((1377978458, -0.7556555169577319, 492688767), (1377978458, 0.4671417847209777, 2107752395), '0.12004560436795493') We are sending mail with results at report@fotonower.com args[1377978456] : ((1377978456, -1.5461356854033614, 492688767), (1377978456, 0.7507933364141532, 2107752395), '0.12004560436795493') We are sending mail with results at report@fotonower.com args[1377978455] : ((1377978455, -2.997019578213948, 492609224), (1377978455, 0.56636843266475, 2107752395), '0.12004560436795493') We are sending mail with results at report@fotonower.com args[1377978453] : ((1377978453, -2.6468294007803936, 492609224), (1377978453, 0.7330431329958064, 2107752395), '0.12004560436795493') We are sending mail with results at report@fotonower.com args[1377978439] : ((1377978439, -1.992817526285214, 492688767), (1377978439, 0.5990966072312655, 2107752395), '0.12004560436795493') We are sending mail with results at report@fotonower.com args[1377978438] : ((1377978438, -2.5077092143196276, 492609224), (1377978438, 0.44996751428949816, 2107752395), '0.12004560436795493') We are sending mail with results at report@fotonower.com args[1377978437] : ((1377978437, -2.5730546744657787, 492609224), (1377978437, 0.4607182600568589, 2107752395), '0.12004560436795493') We are sending mail with results at report@fotonower.com args[1377978414] : ((1377978414, -3.2612411075490337, 492609224), (1377978414, 0.5165502801441317, 2107752395), '0.12004560436795493') We are sending mail with results at report@fotonower.com args[1377978392] : ((1377978392, -2.4501391222103264, 492609224), (1377978392, 0.4783370012941762, 2107752395), '0.12004560436795493') We are sending mail with results at report@fotonower.com args[1377978367] : ((1377978367, -1.740718560442027, 492688767), (1377978367, 0.3947140688218363, 2107752395), '0.12004560436795493') We are sending mail with results at report@fotonower.com args[1377978306] : ((1377978306, -1.8121098176009531, 492688767), (1377978306, 0.4981598178174465, 2107752395), '0.12004560436795493') We are sending mail with results at report@fotonower.com args[1377978298] : ((1377978298, -0.375401293333239, 492688767), (1377978298, 0.4512017953215928, 2107752395), '0.12004560436795493') We are sending mail with results at report@fotonower.com args[1377978294] : ((1377978294, -2.9054487599442713, 492609224), (1377978294, 0.4787148718700771, 2107752395), '0.12004560436795493') We are sending mail with results at report@fotonower.com args[1377978278] : ((1377978278, -2.1157578577321585, 492609224), (1377978278, 0.42882948964953893, 2107752395), '0.12004560436795493') We are sending mail with results at report@fotonower.com args[1377978254] : ((1377978254, 0.9056591519737623, 492688767), (1377978254, 0.5674248509200305, 2107752395), '0.12004560436795493') We are sending mail with results at report@fotonower.com args[1377978253] : ((1377978253, -1.4129297440780777, 492688767), (1377978253, 0.4592972124481705, 2107752395), '0.12004560436795493') We are sending mail with results at report@fotonower.com args[1377978201] : ((1377978201, -2.292379392122436, 492609224), (1377978201, 0.49904435175102896, 2107752395), '0.12004560436795493') We are sending mail with results at report@fotonower.com refus_total : 0.12004560436795493 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=26070739 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26070739_20-08-2025_08_22_25.pdf results_Auto_P26070739_20-08-2025_08_22_25.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26070739_20-08-2025_08_22_25.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','26070739','results_Auto_P26070739_20-08-2025_08_22_25.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26070739_20-08-2025_08_22_25.pdf','pdf','','0.69','0.12004560436795493') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/26070739

https://www.fotonower.com/image?json=false&list_photos_id=1377978543
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1377978541
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1377978539
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1377978536
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1377978486
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1377978467
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1377978458
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1377978456
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1377978455
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1377978453
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1377978439
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1377978438
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1377978437
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1377978414
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1377978392
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1377978367
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1377978306
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1377978298
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1377978294
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1377978278
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1377978254
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1377978253
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1377978201
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 12.00%
Veuillez trouver les photos des contaminants.

exemples de contaminants: autre: https://www.fotonower.com/view/26071060?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/26071063?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/26071065?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/26071066?limit=200
exemples de contaminants: metal: https://www.fotonower.com/view/26071070?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26070739_20-08-2025_08_22_25.pdf.

Lien vers velours :https://www.fotonower.com/velours/26071060,26071061,26071062,26071063,26071064,26071065,26071066,26071067,26071068,26071069,26071070?tags=autre,mal_croppe,background,carton,environnement,pet_clair,papier,pehd,flou,pet_fonce,metal.


L'équipe Fotonower 202 b'' Server: nginx Date: Wed, 20 Aug 2025 06:22:30 GMT Content-Length: 0 Connection: close X-Message-Id: ZK48v5S6Rme3wdVow5GDCw Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1377978543, 1377978541, 1377978539, 1377978536, 1377978486, 1377978467, 1377978458, 1377978456, 1377978455, 1377978453, 1377978439, 1377978438, 1377978437, 1377978414, 1377978392, 1377978367, 1377978306, 1377978298, 1377978294, 1377978278, 1377978254, 1377978253, 1377978201] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978543', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978541', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978539', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978536', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978486', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978467', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978458', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978456', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978455', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978453', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978439', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978438', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978437', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978414', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978392', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978367', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978306', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978298', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978294', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978278', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978254', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978253', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978201', None, None, None, None, None, '3555385') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 23 time used for this insertion : 0.16330313682556152 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.915147066116333 time spend to save output : 0.16372108459472656 total time spend for step 9 : 5.07886815071106 step10:split_time_score Wed Aug 20 08:22:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('07', 23),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 20082025 26070739 Nombre de photos uploadées : 23 / 23040 (0%) 20082025 26070739 Nombre de photos taguées (types de déchets): 0 / 23 (0%) 20082025 26070739 Nombre de photos taguées (volume) : 0 / 23 (0%) elapsed_time : load_data_split_time_score 2.1457672119140625e-06 elapsed_time : order_list_meta_photo_and_scores 5.7220458984375e-06 ??????????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.001081228256225586 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.21522879600524902 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26070736 order by id desc limit 1 Qualite : 0.12004560436795493 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26070739_20-08-2025_08_22_25.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26070739 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26070739 AND mptpi.`type`=3594 To do Qualite : 0.17909196566358027 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26070741_20-08-2025_08_11_46.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26070741 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26070741 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'20082025': {'nb_upload': 23, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1377978543, 1377978541, 1377978539, 1377978536, 1377978486, 1377978467, 1377978458, 1377978456, 1377978455, 1377978453, 1377978439, 1377978438, 1377978437, 1377978414, 1377978392, 1377978367, 1377978306, 1377978298, 1377978294, 1377978278, 1377978254, 1377978253, 1377978201] Looping around the photos to save general results len do output : 1 /26070739Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978543', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978541', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978539', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978536', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978486', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978467', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978458', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978456', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978455', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978453', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978439', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978438', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978437', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978414', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978392', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978367', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978306', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978298', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978294', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978278', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978254', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978253', None, None, None, None, None, '3555385') ('3318', None, None, None, None, None, None, None, '3555385') ('3318', '26070739', '1377978201', None, None, None, None, None, '3555385') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 24 time used for this insertion : 0.022922754287719727 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.5827031135559082 time spend to save output : 0.023162841796875 total time spend for step 10 : 1.6058659553527832 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 23 set_done_treatment 78.98user 28.90system 2:06.40elapsed 85%CPU (0avgtext+0avgdata 3025876maxresident)k 539736inputs+46992outputs (224major+2206500minor)pagefaults 0swaps