python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 1964327 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['3496192'] with mtr_portfolio_ids : ['25867659'] and first list_photo_ids : [] new path : /proc/1964327/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 15 ; length of list_pids : 15 ; length of list_args : 15 time to download the photos : 1.8047518730163574 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Mon Aug 11 08:40:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10957 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-08-11 08:40:34.369830: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-08-11 08:40:34.395302: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493035000 Hz 2025-08-11 08:40:34.396772: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fa794000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-08-11 08:40:34.396822: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-08-11 08:40:34.399312: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-08-11 08:40:34.515899: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x42220830 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-08-11 08:40:34.515930: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-08-11 08:40:34.516787: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-11 08:40:34.517052: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-11 08:40:34.518887: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-11 08:40:34.520925: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-11 08:40:34.521214: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-11 08:40:34.523232: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-11 08:40:34.524209: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-11 08:40:34.528272: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-11 08:40:34.529695: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-11 08:40:34.529765: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-11 08:40:34.530505: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-08-11 08:40:34.530520: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-08-11 08:40:34.530528: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-08-11 08:40:34.531829: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10011 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-08-11 08:40:34.776606: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-11 08:40:34.776717: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-11 08:40:34.776733: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-11 08:40:34.776748: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-11 08:40:34.776762: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-11 08:40:34.776776: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-11 08:40:34.776789: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-11 08:40:34.776803: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-11 08:40:34.777998: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-11 08:40:34.779208: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-11 08:40:34.779241: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-11 08:40:34.779257: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-11 08:40:34.779271: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-11 08:40:34.779284: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-11 08:40:34.779298: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-11 08:40:34.779311: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-11 08:40:34.779325: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-11 08:40:34.780520: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-11 08:40:34.780553: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-08-11 08:40:34.780561: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-08-11 08:40:34.780568: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-08-11 08:40:34.781803: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10011 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-08-11 08:40:43.206863: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-11 08:40:43.448905: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 15 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 44.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 47.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 44.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 39.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 48.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 41.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 47.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 48.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 47.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 39.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 63.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 64.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 49.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 46.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 60.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 9 Detection mask done ! Trying to reset tf kernel 1965102 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1089 tf kernel not reseted sub process len(results) : 15 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 15 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1888 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0023398399353027344 nb_pixel_total : 119083 time to create 1 rle with old method : 0.1274728775024414 length of segment : 572 time for calcul the mask position with numpy : 0.0664520263671875 nb_pixel_total : 964139 time to create 1 rle with new method : 0.1956319808959961 length of segment : 1650 time for calcul the mask position with numpy : 0.00018358230590820312 nb_pixel_total : 4837 time to create 1 rle with old method : 0.005474567413330078 length of segment : 108 time for calcul the mask position with numpy : 0.012433052062988281 nb_pixel_total : 725633 time to create 1 rle with new method : 0.06748485565185547 length of segment : 994 time for calcul the mask position with numpy : 0.0002856254577636719 nb_pixel_total : 6764 time to create 1 rle with old method : 0.0076634883880615234 length of segment : 95 time for calcul the mask position with numpy : 0.0039904117584228516 nb_pixel_total : 169321 time to create 1 rle with new method : 0.007009029388427734 length of segment : 629 time for calcul the mask position with numpy : 0.000164031982421875 nb_pixel_total : 1901 time to create 1 rle with old method : 0.002206563949584961 length of segment : 61 time for calcul the mask position with numpy : 0.0004153251647949219 nb_pixel_total : 9090 time to create 1 rle with old method : 0.010488510131835938 length of segment : 143 time for calcul the mask position with numpy : 0.00026798248291015625 nb_pixel_total : 7633 time to create 1 rle with old method : 0.008566617965698242 length of segment : 94 time for calcul the mask position with numpy : 0.0003256797790527344 nb_pixel_total : 11702 time to create 1 rle with old method : 0.012998819351196289 length of segment : 107 time for calcul the mask position with numpy : 0.00025272369384765625 nb_pixel_total : 11752 time to create 1 rle with old method : 0.013243675231933594 length of segment : 137 time for calcul the mask position with numpy : 0.0003209114074707031 nb_pixel_total : 7030 time to create 1 rle with old method : 0.00783991813659668 length of segment : 131 time for calcul the mask position with numpy : 0.0004220008850097656 nb_pixel_total : 10117 time to create 1 rle with old method : 0.011549234390258789 length of segment : 124 time for calcul the mask position with numpy : 0.00033354759216308594 nb_pixel_total : 8867 time to create 1 rle with old method : 0.00988149642944336 length of segment : 101 time for calcul the mask position with numpy : 0.0007011890411376953 nb_pixel_total : 28364 time to create 1 rle with old method : 0.032578229904174805 length of segment : 204 time for calcul the mask position with numpy : 0.017813920974731445 nb_pixel_total : 733638 time to create 1 rle with new method : 0.036248207092285156 length of segment : 984 time for calcul the mask position with numpy : 0.002898693084716797 nb_pixel_total : 89068 time to create 1 rle with old method : 0.10300111770629883 length of segment : 603 time for calcul the mask position with numpy : 0.00037097930908203125 nb_pixel_total : 6333 time to create 1 rle with old method : 0.007597446441650391 length of segment : 181 time for calcul the mask position with numpy : 0.0006971359252929688 nb_pixel_total : 22435 time to create 1 rle with old method : 0.035964012145996094 length of segment : 193 time for calcul the mask position with numpy : 0.025937557220458984 nb_pixel_total : 1175039 time to create 1 rle with new method : 0.06532430648803711 length of segment : 1806 time for calcul the mask position with numpy : 0.0001823902130126953 nb_pixel_total : 2125 time to create 1 rle with old method : 0.002530813217163086 length of segment : 35 time for calcul the mask position with numpy : 0.0002589225769042969 nb_pixel_total : 6483 time to create 1 rle with old method : 0.0074956417083740234 length of segment : 89 time for calcul the mask position with numpy : 0.00022125244140625 nb_pixel_total : 6009 time to create 1 rle with old method : 0.0067746639251708984 length of segment : 74 time for calcul the mask position with numpy : 0.002721071243286133 nb_pixel_total : 111417 time to create 1 rle with old method : 0.11662125587463379 length of segment : 528 time for calcul the mask position with numpy : 0.07675671577453613 nb_pixel_total : 1480418 time to create 1 rle with new method : 0.11361193656921387 length of segment : 1382 time for calcul the mask position with numpy : 0.00043773651123046875 nb_pixel_total : 8693 time to create 1 rle with old method : 0.010103464126586914 length of segment : 128 time for calcul the mask position with numpy : 0.0002925395965576172 nb_pixel_total : 4691 time to create 1 rle with old method : 0.005513668060302734 length of segment : 95 time for calcul the mask position with numpy : 0.0005600452423095703 nb_pixel_total : 21055 time to create 1 rle with old method : 0.023584842681884766 length of segment : 131 time for calcul the mask position with numpy : 0.0003223419189453125 nb_pixel_total : 10660 time to create 1 rle with old method : 0.011781930923461914 length of segment : 94 time for calcul the mask position with numpy : 0.027065038681030273 nb_pixel_total : 1492160 time to create 1 rle with new method : 0.09781026840209961 length of segment : 1402 time for calcul the mask position with numpy : 0.0005006790161132812 nb_pixel_total : 11650 time to create 1 rle with old method : 0.013410329818725586 length of segment : 123 time for calcul the mask position with numpy : 0.0004570484161376953 nb_pixel_total : 8834 time to create 1 rle with old method : 0.010362386703491211 length of segment : 173 time for calcul the mask position with numpy : 0.0005128383636474609 nb_pixel_total : 16450 time to create 1 rle with old method : 0.018741846084594727 length of segment : 167 time for calcul the mask position with numpy : 0.0003523826599121094 nb_pixel_total : 13234 time to create 1 rle with old method : 0.015024423599243164 length of segment : 284 time for calcul the mask position with numpy : 0.0004482269287109375 nb_pixel_total : 13139 time to create 1 rle with old method : 0.015309572219848633 length of segment : 290 time for calcul the mask position with numpy : 0.015201330184936523 nb_pixel_total : 713546 time to create 1 rle with new method : 0.07062721252441406 length of segment : 943 time for calcul the mask position with numpy : 0.0008132457733154297 nb_pixel_total : 16330 time to create 1 rle with old method : 0.02603912353515625 length of segment : 196 time for calcul the mask position with numpy : 0.019188404083251953 nb_pixel_total : 923833 time to create 1 rle with new method : 0.051816463470458984 length of segment : 1869 time for calcul the mask position with numpy : 0.00012803077697753906 nb_pixel_total : 5812 time to create 1 rle with old method : 0.006867170333862305 length of segment : 86 time for calcul the mask position with numpy : 0.0015437602996826172 nb_pixel_total : 85172 time to create 1 rle with old method : 0.0977470874786377 length of segment : 535 time for calcul the mask position with numpy : 0.00023436546325683594 nb_pixel_total : 11105 time to create 1 rle with old method : 0.013074159622192383 length of segment : 105 time spent for convertir_results : 4.932225227355957 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 41 chid ids of type : 3594 Number RLEs to save : 17646 save missing photos in datou_result : time spend for datou_step_exec : 31.84683609008789 time spend to save output : 1.5182445049285889 total time spend for step 1 : 33.36508059501648 step2:crop_condition Mon Aug 11 08:41:05 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 15 ! batch 1 Loaded 41 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 20 About to insert : list_path_to_insert length 20 new photo from crops ! About to upload 20 photos upload in portfolio : 3736932 init cache_photo without model_param we have 20 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1754894469_1964327 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 20 photos in the portfolio 3736932 time of upload the photos Elapsed time : 5.621235609054565 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 3736932 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1754894478_1964327 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 3 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.2541401386260986 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 17 About to insert : list_path_to_insert length 17 new photo from crops ! About to upload 17 photos upload in portfolio : 3736932 init cache_photo without model_param we have 17 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1754894510_1964327 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 17 photos in the portfolio 3736932 time of upload the photos Elapsed time : 5.098836660385132 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1754894516_1964327 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.7229504585266113 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1376394333, 1376394330, 1376394327, 1376394325, 1376394306, 1376394304, 1376394301, 1376394297, 1376394292, 1376394289, 1376394265, 1376394262, 1376394260, 1376394256, 1376394253] Looping around the photos to save general results len do output : 41 /1376400523Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400524Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400525Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400526Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400527Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400528Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400529Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400530Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400531Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400532Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400533Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400534Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400535Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400536Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400537Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400538Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400539Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400540Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400541Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400542Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400546Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400547Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400548Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400574Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400575Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400576Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400577Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400578Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400579Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400580Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400581Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400582Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400583Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400584Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400585Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400586Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400587Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400588Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400589Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400590Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1376400593Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394333', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394330', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394327', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394325', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394306', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394304', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394301', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394297', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394292', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394289', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394265', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394262', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394260', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394256', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394253', None, None, None, None, None, '3496192') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 138 time used for this insertion : 0.1626443862915039 save_final save missing photos in datou_result : time spend for datou_step_exec : 51.36532664299011 time spend to save output : 0.16434621810913086 total time spend for step 2 : 51.52967286109924 step3:rle_unique_nms_with_priority Mon Aug 11 08:41:56 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 41 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 0.572613000869751 time for calcul the mask position with numpy : 0.015198707580566406 nb_pixel_total : 718435 time to create 1 rle with new method : 0.22960877418518066 time for calcul the mask position with numpy : 0.008734941482543945 nb_pixel_total : 270692 time to create 1 rle with new method : 0.12913227081298828 time for calcul the mask position with numpy : 0.007837295532226562 nb_pixel_total : 2655 time to create 1 rle with old method : 0.0032677650451660156 time for calcul the mask position with numpy : 0.017769813537597656 nb_pixel_total : 962735 time to create 1 rle with new method : 0.24498820304870605 time for calcul the mask position with numpy : 0.007216691970825195 nb_pixel_total : 119083 time to create 1 rle with old method : 0.13141870498657227 create new chi : 0.8212418556213379 time to delete rle : 0.029988765716552734 batch 1 Loaded 9 chid ids of type : 3594 +++++++Number RLEs to save : 6785 TO DO : save crop sub photo not yet done ! save time : 0.5270330905914307 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.04727029800415039 time for calcul the mask position with numpy : 0.12329816818237305 nb_pixel_total : 2066836 time to create 1 rle with new method : 0.09073019027709961 time for calcul the mask position with numpy : 0.005922079086303711 nb_pixel_total : 6764 time to create 1 rle with old method : 0.007132053375244141 create new chi : 0.2272939682006836 time to delete rle : 0.00025343894958496094 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 1270 TO DO : save crop sub photo not yet done ! save time : 0.13962817192077637 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.039542436599731445 time for calcul the mask position with numpy : 0.017832279205322266 nb_pixel_total : 1904279 time to create 1 rle with new method : 0.1095130443572998 time for calcul the mask position with numpy : 0.0067806243896484375 nb_pixel_total : 169321 time to create 1 rle with new method : 0.02676868438720703 create new chi : 0.16627240180969238 time to delete rle : 0.0003268718719482422 batch 1 Loaded 3 chid ids of type : 3594 ++Number RLEs to save : 2338 TO DO : save crop sub photo not yet done ! save time : 0.2199876308441162 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.18448209762573242 time for calcul the mask position with numpy : 0.0740518569946289 nb_pixel_total : 2062609 time to create 1 rle with new method : 0.19152379035949707 time for calcul the mask position with numpy : 0.006211519241333008 nb_pixel_total : 9090 time to create 1 rle with old method : 0.010221004486083984 time for calcul the mask position with numpy : 0.0060214996337890625 nb_pixel_total : 1901 time to create 1 rle with old method : 0.002155780792236328 create new chi : 0.30051589012145996 time to delete rle : 0.00033473968505859375 batch 1 Loaded 5 chid ids of type : 3594 +++Number RLEs to save : 1488 TO DO : save crop sub photo not yet done ! save time : 0.1765458583831787 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.2415456771850586 time for calcul the mask position with numpy : 0.08809876441955566 nb_pixel_total : 2042513 time to create 1 rle with new method : 0.27253246307373047 time for calcul the mask position with numpy : 0.0063686370849609375 nb_pixel_total : 11752 time to create 1 rle with old method : 0.01331186294555664 time for calcul the mask position with numpy : 0.00621795654296875 nb_pixel_total : 11702 time to create 1 rle with old method : 0.013194799423217773 time for calcul the mask position with numpy : 0.006299734115600586 nb_pixel_total : 7633 time to create 1 rle with old method : 0.008572578430175781 create new chi : 0.42531704902648926 time to delete rle : 0.00031304359436035156 batch 1 Loaded 7 chid ids of type : 3594 +++Number RLEs to save : 1756 TO DO : save crop sub photo not yet done ! save time : 0.17197942733764648 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.03909659385681152 time for calcul the mask position with numpy : 0.15462040901184082 nb_pixel_total : 2056453 time to create 1 rle with new method : 0.2095201015472412 time for calcul the mask position with numpy : 0.006030559539794922 nb_pixel_total : 10117 time to create 1 rle with old method : 0.010902166366577148 time for calcul the mask position with numpy : 0.005979061126708984 nb_pixel_total : 7030 time to create 1 rle with old method : 0.0073223114013671875 create new chi : 0.4019908905029297 time to delete rle : 0.0002834796905517578 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 1590 TO DO : save crop sub photo not yet done ! save time : 0.14611005783081055 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.31429338455200195 time for calcul the mask position with numpy : 0.20616483688354492 nb_pixel_total : 2064733 time to create 1 rle with new method : 0.11909985542297363 time for calcul the mask position with numpy : 0.0063741207122802734 nb_pixel_total : 8867 time to create 1 rle with old method : 0.009942054748535156 create new chi : 0.3523597717285156 time to delete rle : 0.0002243518829345703 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 1282 TO DO : save crop sub photo not yet done ! save time : 0.14288878440856934 nb_obj : 3 nb_hashtags : 1 time to prepare the origin masks : 0.22969460487365723 time for calcul the mask position with numpy : 0.11180782318115234 nb_pixel_total : 1232331 time to create 1 rle with new method : 0.09452700614929199 time for calcul the mask position with numpy : 0.006891727447509766 nb_pixel_total : 79267 time to create 1 rle with old method : 0.08667588233947754 time for calcul the mask position with numpy : 0.012113332748413086 nb_pixel_total : 733638 time to create 1 rle with new method : 0.10387563705444336 time for calcul the mask position with numpy : 0.00632476806640625 nb_pixel_total : 28364 time to create 1 rle with old method : 0.031019210815429688 create new chi : 0.463486909866333 time to delete rle : 0.0006773471832275391 batch 1 Loaded 7 chid ids of type : 3594 +++++Number RLEs to save : 4517 TO DO : save crop sub photo not yet done ! save time : 0.388641357421875 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.06634998321533203 time for calcul the mask position with numpy : 0.011659860610961914 nb_pixel_total : 893914 time to create 1 rle with new method : 0.19520092010498047 time for calcul the mask position with numpy : 0.04328036308288574 nb_pixel_total : 1150918 time to create 1 rle with new method : 0.16300487518310547 time for calcul the mask position with numpy : 0.006388187408447266 nb_pixel_total : 22435 time to create 1 rle with old method : 0.024313926696777344 time for calcul the mask position with numpy : 0.006040811538696289 nb_pixel_total : 6333 time to create 1 rle with old method : 0.006821632385253906 create new chi : 0.47040510177612305 time to delete rle : 0.0007781982421875 batch 1 Loaded 7 chid ids of type : 3594 +++++++++Number RLEs to save : 5328 TO DO : save crop sub photo not yet done ! save time : 0.4442605972290039 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.03217458724975586 time for calcul the mask position with numpy : 0.01933741569519043 nb_pixel_total : 2071475 time to create 1 rle with new method : 0.15048909187316895 time for calcul the mask position with numpy : 0.006118059158325195 nb_pixel_total : 2125 time to create 1 rle with old method : 0.0023584365844726562 create new chi : 0.18545079231262207 time to delete rle : 0.0002124309539794922 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 1150 TO DO : save crop sub photo not yet done ! save time : 0.13859343528747559 nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 0.624514102935791 time for calcul the mask position with numpy : 0.009683609008789062 nb_pixel_total : 481765 time to create 1 rle with new method : 0.10849308967590332 time for calcul the mask position with numpy : 0.059072256088256836 nb_pixel_total : 1467926 time to create 1 rle with new method : 0.19358563423156738 time for calcul the mask position with numpy : 0.0071010589599609375 nb_pixel_total : 111417 time to create 1 rle with old method : 0.12222695350646973 time for calcul the mask position with numpy : 0.006868839263916016 nb_pixel_total : 6009 time to create 1 rle with old method : 0.006818294525146484 time for calcul the mask position with numpy : 0.006628513336181641 nb_pixel_total : 6483 time to create 1 rle with old method : 0.007215738296508789 create new chi : 0.5435402393341064 time to delete rle : 0.0009500980377197266 batch 1 Loaded 9 chid ids of type : 3594 +++++Number RLEs to save : 5226 TO DO : save crop sub photo not yet done ! save time : 0.3519127368927002 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.17123961448669434 time for calcul the mask position with numpy : 0.1476759910583496 nb_pixel_total : 2039161 time to create 1 rle with new method : 0.09093236923217773 time for calcul the mask position with numpy : 0.006319284439086914 nb_pixel_total : 21055 time to create 1 rle with old method : 0.023454666137695312 time for calcul the mask position with numpy : 0.006109952926635742 nb_pixel_total : 4691 time to create 1 rle with old method : 0.005282402038574219 time for calcul the mask position with numpy : 0.007193803787231445 nb_pixel_total : 8693 time to create 1 rle with old method : 0.009364604949951172 create new chi : 0.3054349422454834 time to delete rle : 0.0003070831298828125 batch 1 Loaded 7 chid ids of type : 3594 +++Number RLEs to save : 1788 TO DO : save crop sub photo not yet done ! save time : 0.17575693130493164 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.17340946197509766 time for calcul the mask position with numpy : 0.009768962860107422 nb_pixel_total : 581440 time to create 1 rle with new method : 0.17183470726013184 time for calcul the mask position with numpy : 0.05758523941040039 nb_pixel_total : 1481500 time to create 1 rle with new method : 0.15314769744873047 time for calcul the mask position with numpy : 0.006487131118774414 nb_pixel_total : 10660 time to create 1 rle with old method : 0.01160287857055664 create new chi : 0.42558884620666504 time to delete rle : 0.0006382465362548828 batch 1 Loaded 5 chid ids of type : 3594 +++Number RLEs to save : 4072 TO DO : save crop sub photo not yet done ! save time : 0.33289361000061035 nb_obj : 6 nb_hashtags : 2 time to prepare the origin masks : 0.5052840709686279 time for calcul the mask position with numpy : 0.052255868911743164 nb_pixel_total : 1296747 time to create 1 rle with new method : 0.32166194915771484 time for calcul the mask position with numpy : 0.01106119155883789 nb_pixel_total : 713546 time to create 1 rle with new method : 0.26580262184143066 time for calcul the mask position with numpy : 0.006233692169189453 nb_pixel_total : 13139 time to create 1 rle with old method : 0.014193058013916016 time for calcul the mask position with numpy : 0.006273746490478516 nb_pixel_total : 13234 time to create 1 rle with old method : 0.0145721435546875 time for calcul the mask position with numpy : 0.006654024124145508 nb_pixel_total : 16450 time to create 1 rle with old method : 0.017772674560546875 time for calcul the mask position with numpy : 0.00616002082824707 nb_pixel_total : 8834 time to create 1 rle with old method : 0.009746074676513672 time for calcul the mask position with numpy : 0.005934476852416992 nb_pixel_total : 11650 time to create 1 rle with old method : 0.012413978576660156 create new chi : 0.7611298561096191 time to delete rle : 0.0007498264312744141 batch 1 Loaded 13 chid ids of type : 3594 ++++++Number RLEs to save : 5040 TO DO : save crop sub photo not yet done ! save time : 0.447521448135376 nb_obj : 5 nb_hashtags : 2 time to prepare the origin masks : 0.0787045955657959 time for calcul the mask position with numpy : 0.01319575309753418 nb_pixel_total : 1050366 time to create 1 rle with new method : 0.0634298324584961 time for calcul the mask position with numpy : 0.0062754154205322266 nb_pixel_total : 11105 time to create 1 rle with old method : 0.011943340301513672 time for calcul the mask position with numpy : 0.007119417190551758 nb_pixel_total : 85003 time to create 1 rle with old method : 0.09776544570922852 time for calcul the mask position with numpy : 0.00599980354309082 nb_pixel_total : 3293 time to create 1 rle with old method : 0.0036706924438476562 time for calcul the mask position with numpy : 0.012734651565551758 nb_pixel_total : 907503 time to create 1 rle with new method : 0.21376824378967285 time for calcul the mask position with numpy : 0.006033658981323242 nb_pixel_total : 16330 time to create 1 rle with old method : 0.017916202545166016 create new chi : 0.46930813789367676 time to delete rle : 0.0010535717010498047 batch 1 Loaded 11 chid ids of type : 3594 ++++++++++Number RLEs to save : 6556 TO DO : save crop sub photo not yet done ! save time : 0.5272130966186523 map_output_result : {1376394333: (0.0, 'Should be the crop_list due to order', 0), 1376394330: (0.0, 'Should be the crop_list due to order', 0), 1376394327: (0.0, 'Should be the crop_list due to order', 0), 1376394325: (0.0, 'Should be the crop_list due to order', 0), 1376394306: (0.0, 'Should be the crop_list due to order', 0), 1376394304: (0.0, 'Should be the crop_list due to order', 0), 1376394301: (0.0, 'Should be the crop_list due to order', 0), 1376394297: (0.0, 'Should be the crop_list due to order', 0), 1376394292: (0.0, 'Should be the crop_list due to order', 0), 1376394289: (0.0, 'Should be the crop_list due to order', 0), 1376394265: (0.0, 'Should be the crop_list due to order', 0), 1376394262: (0.0, 'Should be the crop_list due to order', 0), 1376394260: (0.0, 'Should be the crop_list due to order', 0), 1376394256: (0.0, 'Should be the crop_list due to order', 0), 1376394253: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1376394333, 1376394330, 1376394327, 1376394325, 1376394306, 1376394304, 1376394301, 1376394297, 1376394292, 1376394289, 1376394265, 1376394262, 1376394260, 1376394256, 1376394253] Looping around the photos to save general results len do output : 15 /1376394333.Didn't retrieve data . /1376394330.Didn't retrieve data . /1376394327.Didn't retrieve data . /1376394325.Didn't retrieve data . /1376394306.Didn't retrieve data . /1376394304.Didn't retrieve data . /1376394301.Didn't retrieve data . /1376394297.Didn't retrieve data . /1376394292.Didn't retrieve data . /1376394289.Didn't retrieve data . /1376394265.Didn't retrieve data . /1376394262.Didn't retrieve data . /1376394260.Didn't retrieve data . /1376394256.Didn't retrieve data . /1376394253.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394333', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394330', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394327', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394325', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394306', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394304', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394301', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394297', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394292', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394289', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394265', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394262', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394260', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394256', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394253', None, None, None, None, None, '3496192') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 45 time used for this insertion : 0.017606258392333984 save_final save missing photos in datou_result : time spend for datou_step_exec : 14.538946390151978 time spend to save output : 0.018022537231445312 total time spend for step 3 : 14.556968927383423 step4:ventilate_hashtags_in_portfolio Mon Aug 11 08:42:11 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 25867659 get user id for portfolio 25867659 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=25867659 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('mal_croppe','carton','pet_fonce','background','metal','autre','pehd','papier','environnement','pet_clair','flou')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=25867659 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('mal_croppe','carton','pet_fonce','background','metal','autre','pehd','papier','environnement','pet_clair','flou')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=25867659 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('mal_croppe','carton','pet_fonce','background','metal','autre','pehd','papier','environnement','pet_clair','flou')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/25867949,25867950,25867951,25867952,25867953,25867954,25867955,25867956,25867957,25867958,25867959?tags=mal_croppe,carton,pet_fonce,background,metal,autre,pehd,papier,environnement,pet_clair,flou Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1376394333, 1376394330, 1376394327, 1376394325, 1376394306, 1376394304, 1376394301, 1376394297, 1376394292, 1376394289, 1376394265, 1376394262, 1376394260, 1376394256, 1376394253] Looping around the photos to save general results len do output : 1 /25867659. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394333', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394330', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394327', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394325', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394306', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394304', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394301', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394297', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394292', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394289', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394265', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394262', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394260', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394256', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394253', None, None, None, None, None, '3496192') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 16 time used for this insertion : 0.02001500129699707 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.7404868602752686 time spend to save output : 0.02045440673828125 total time spend for step 4 : 1.7609412670135498 step5:final Mon Aug 11 08:42:12 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1376394333: ('0.2745930748456791',), 1376394330: ('0.2745930748456791',), 1376394327: ('0.2745930748456791',), 1376394325: ('0.2745930748456791',), 1376394306: ('0.2745930748456791',), 1376394304: ('0.2745930748456791',), 1376394301: ('0.2745930748456791',), 1376394297: ('0.2745930748456791',), 1376394292: ('0.2745930748456791',), 1376394289: ('0.2745930748456791',), 1376394265: ('0.2745930748456791',), 1376394262: ('0.2745930748456791',), 1376394260: ('0.2745930748456791',), 1376394256: ('0.2745930748456791',), 1376394253: ('0.2745930748456791',)} new output for save of step final : {1376394333: ('0.2745930748456791',), 1376394330: ('0.2745930748456791',), 1376394327: ('0.2745930748456791',), 1376394325: ('0.2745930748456791',), 1376394306: ('0.2745930748456791',), 1376394304: ('0.2745930748456791',), 1376394301: ('0.2745930748456791',), 1376394297: ('0.2745930748456791',), 1376394292: ('0.2745930748456791',), 1376394289: ('0.2745930748456791',), 1376394265: ('0.2745930748456791',), 1376394262: ('0.2745930748456791',), 1376394260: ('0.2745930748456791',), 1376394256: ('0.2745930748456791',), 1376394253: ('0.2745930748456791',)} [1376394333, 1376394330, 1376394327, 1376394325, 1376394306, 1376394304, 1376394301, 1376394297, 1376394292, 1376394289, 1376394265, 1376394262, 1376394260, 1376394256, 1376394253] Looping around the photos to save general results len do output : 15 /1376394333.Didn't retrieve data . /1376394330.Didn't retrieve data . /1376394327.Didn't retrieve data . /1376394325.Didn't retrieve data . /1376394306.Didn't retrieve data . /1376394304.Didn't retrieve data . /1376394301.Didn't retrieve data . /1376394297.Didn't retrieve data . /1376394292.Didn't retrieve data . /1376394289.Didn't retrieve data . /1376394265.Didn't retrieve data . /1376394262.Didn't retrieve data . /1376394260.Didn't retrieve data . /1376394256.Didn't retrieve data . /1376394253.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394333', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394330', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394327', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394325', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394306', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394304', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394301', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394297', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394292', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394289', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394265', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394262', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394260', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394256', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394253', None, None, None, None, None, '3496192') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 45 time used for this insertion : 0.018965482711791992 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.12539005279541016 time spend to save output : 0.019643306732177734 total time spend for step 5 : 0.1450333595275879 step6:blur_detection Mon Aug 11 08:42:13 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1754894429_1964327_1376394333_adb12429fe61ed91c45138fd452e798b.jpg resize: (1080, 1920) 1376394333 -0.06997299114849541 treat image : temp/1754894429_1964327_1376394330_f7f1dfa1cf4687c48a060ba70ebdaab0.jpg resize: (1080, 1920) 1376394330 0.8121591041532854 treat image : temp/1754894429_1964327_1376394327_5817ce15f9f853f7457f21d76e8011a4.jpg resize: (1080, 1920) 1376394327 0.7913162251184331 treat image : temp/1754894429_1964327_1376394325_d023545f5d488b6edd76fd9fd3925f8d.jpg resize: (1080, 1920) 1376394325 1.0179652203596479 treat image : temp/1754894429_1964327_1376394306_249032193d7b38e1900507dea8155161.jpg resize: (1080, 1920) 1376394306 1.3536295339085098 treat image : temp/1754894429_1964327_1376394304_51c8f2f0ac8434b9161b3d8f00cb8bde.jpg resize: (1080, 1920) 1376394304 5.1140010446564474 treat image : temp/1754894429_1964327_1376394301_3dea6e8919d18c72d7a78a8f6cc97515.jpg resize: (1080, 1920) 1376394301 -0.10098682489087217 treat image : temp/1754894429_1964327_1376394297_7173e552cbfa7ba478abe879349fff6b.jpg resize: (1080, 1920) 1376394297 5.0310345206761 treat image : temp/1754894429_1964327_1376394292_57f47de61c4b9cd1afb0f939e23a0754.jpg resize: (1080, 1920) 1376394292 2.01177056274147 treat image : temp/1754894429_1964327_1376394289_7650914cbc7e44b13000eb799f22bff6.jpg resize: (1080, 1920) 1376394289 1.1665181516534016 treat image : temp/1754894429_1964327_1376394265_3929f859b20f0fbc54baba4bb9c5746c.jpg resize: (1080, 1920) 1376394265 3.275973894165184 treat image : temp/1754894429_1964327_1376394262_277dcc8ec2a8f0b2b94dbb209540e4b9.jpg resize: (1080, 1920) 1376394262 1.576227242552276 treat image : temp/1754894429_1964327_1376394260_6703a01f781df4d2217368af0babf4e6.jpg resize: (1080, 1920) 1376394260 3.231447724769417 treat image : temp/1754894429_1964327_1376394256_7d879e5217e117e6f37b3e66916c3061.jpg resize: (1080, 1920) 1376394256 1.1668438586351026 treat image : temp/1754894429_1964327_1376394253_0c49a009419f2781ad5e0704a1ccc2b6.jpg resize: (1080, 1920) 1376394253 2.513552909865967 treat image : temp/1754894429_1964327_1376394333_adb12429fe61ed91c45138fd452e798b_rle_crop_3910817854_0.png resize: (968, 1153) 1376400523 0.7260669357548066 treat image : temp/1754894429_1964327_1376394333_adb12429fe61ed91c45138fd452e798b_rle_crop_3910817853_0.png resize: (108, 121) 1376400524 -1.4257470729013804 treat image : temp/1754894429_1964327_1376394330_f7f1dfa1cf4687c48a060ba70ebdaab0_rle_crop_3910817855_0.png resize: (78, 139) 1376400525 -1.1183817910452212 treat image : temp/1754894429_1964327_1376394325_d023545f5d488b6edd76fd9fd3925f8d_rle_crop_3910817857_0.png resize: (61, 50) 1376400526 -2.394702081547182 treat image : temp/1754894429_1964327_1376394306_249032193d7b38e1900507dea8155161_rle_crop_3910817859_0.png resize: (94, 117) 1376400527 -0.07030178696753535 treat image : temp/1754894429_1964327_1376394306_249032193d7b38e1900507dea8155161_rle_crop_3910817860_0.png resize: (102, 133) 1376400528 1.6671222695145143 treat image : temp/1754894429_1964327_1376394304_51c8f2f0ac8434b9161b3d8f00cb8bde_rle_crop_3910817862_0.png resize: (131, 81) 1376400529 -0.27401621798158954 treat image : temp/1754894429_1964327_1376394301_3dea6e8919d18c72d7a78a8f6cc97515_rle_crop_3910817864_0.png resize: (101, 116) 1376400530 -0.7664587344453383 treat image : temp/1754894429_1964327_1376394292_57f47de61c4b9cd1afb0f939e23a0754_rle_crop_3910817868_0.png resize: (107, 168) 1376400531 -1.983835561019035 treat image : temp/1754894429_1964327_1376394289_7650914cbc7e44b13000eb799f22bff6_rle_crop_3910817871_0.png resize: (35, 73) 1376400532 4.035322484536544 treat image : temp/1754894429_1964327_1376394265_3929f859b20f0fbc54baba4bb9c5746c_rle_crop_3910817872_0.png resize: (89, 109) 1376400533 -0.23559249928944134 treat image : temp/1754894429_1964327_1376394265_3929f859b20f0fbc54baba4bb9c5746c_rle_crop_3910817873_0.png resize: (74, 107) 1376400534 0.24042606382894985 treat image : temp/1754894429_1964327_1376394262_277dcc8ec2a8f0b2b94dbb209540e4b9_rle_crop_3910817877_0.png resize: (95, 77) 1376400535 -0.5661342029318113 treat image : temp/1754894429_1964327_1376394262_277dcc8ec2a8f0b2b94dbb209540e4b9_rle_crop_3910817876_0.png resize: (119, 145) 1376400536 -0.7510442603492439 treat image : temp/1754894429_1964327_1376394260_6703a01f781df4d2217368af0babf4e6_rle_crop_3910817879_0.png resize: (94, 177) 1376400537 0.12849009902855782 treat image : temp/1754894429_1964327_1376394256_7d879e5217e117e6f37b3e66916c3061_rle_crop_3910817884_0.png resize: (185, 111) 1376400538 -1.9277523147013604 treat image : temp/1754894429_1964327_1376394256_7d879e5217e117e6f37b3e66916c3061_rle_crop_3910817883_0.png resize: (165, 118) 1376400539 0.6771144869388824 treat image : temp/1754894429_1964327_1376394256_7d879e5217e117e6f37b3e66916c3061_rle_crop_3910817885_0.png resize: (178, 142) 1376400540 -1.9924628983904402 treat image : temp/1754894429_1964327_1376394256_7d879e5217e117e6f37b3e66916c3061_rle_crop_3910817882_0.png resize: (170, 123) 1376400541 -1.6794194100222397 treat image : temp/1754894429_1964327_1376394253_0c49a009419f2781ad5e0704a1ccc2b6_rle_crop_3910817891_0.png resize: (101, 157) 1376400542 -1.3044534724918393 treat image : temp/1754894429_1964327_1376394325_d023545f5d488b6edd76fd9fd3925f8d_rle_crop_3910817858_0.png resize: (135, 103) 1376400546 -1.3039941631086625 treat image : temp/1754894429_1964327_1376394256_7d879e5217e117e6f37b3e66916c3061_rle_crop_3910817886_0.png resize: (939, 1007) 1376400547 1.5082365286208268 treat image : temp/1754894429_1964327_1376394256_7d879e5217e117e6f37b3e66916c3061_rle_crop_3910817881_0.png resize: (123, 143) 1376400548 -0.790623406338348 treat image : temp/1754894429_1964327_1376394333_adb12429fe61ed91c45138fd452e798b_rle_crop_3910817852_0.png resize: (1002, 1469) 1376400574 -1.573629335734455 treat image : temp/1754894429_1964327_1376394333_adb12429fe61ed91c45138fd452e798b_rle_crop_3910817851_0.png resize: (530, 334) 1376400575 -0.45328770889231945 treat image : temp/1754894429_1964327_1376394327_5817ce15f9f853f7457f21d76e8011a4_rle_crop_3910817856_0.png resize: (517, 492) 1376400576 -1.038936170606374 treat image : temp/1754894429_1964327_1376394306_249032193d7b38e1900507dea8155161_rle_crop_3910817861_0.png resize: (137, 121) 1376400577 -1.9977075724175066 treat image : temp/1754894429_1964327_1376394297_7173e552cbfa7ba478abe879349fff6b_rle_crop_3910817866_0.png resize: (970, 1103) 1376400578 1.6170535922554046 treat image : temp/1754894429_1964327_1376394297_7173e552cbfa7ba478abe879349fff6b_rle_crop_3910817867_0.png resize: (486, 647) 1376400579 -0.9482051191984179 treat image : temp/1754894429_1964327_1376394297_7173e552cbfa7ba478abe879349fff6b_rle_crop_3910817865_0.png resize: (193, 204) 1376400580 -0.09526673266130246 treat image : temp/1754894429_1964327_1376394292_57f47de61c4b9cd1afb0f939e23a0754_rle_crop_3910817870_0.png resize: (1009, 1453) 1376400581 -0.5296647443982532 treat image : temp/1754894429_1964327_1376394292_57f47de61c4b9cd1afb0f939e23a0754_rle_crop_3910817869_0.png resize: (184, 159) 1376400582 -1.6588696699272902 treat image : temp/1754894429_1964327_1376394265_3929f859b20f0fbc54baba4bb9c5746c_rle_crop_3910817875_0.png resize: (1006, 1656) 1376400583 0.17853321956967172 treat image : temp/1754894429_1964327_1376394265_3929f859b20f0fbc54baba4bb9c5746c_rle_crop_3910817874_0.png resize: (522, 346) 1376400584 0.23965578594612771 treat image : temp/1754894429_1964327_1376394262_277dcc8ec2a8f0b2b94dbb209540e4b9_rle_crop_3910817878_0.png resize: (127, 208) 1376400585 0.06394538025178026 treat image : temp/1754894429_1964327_1376394260_6703a01f781df4d2217368af0babf4e6_rle_crop_3910817880_0.png resize: (1010, 1640) 1376400586 0.2360121851987789 treat image : temp/1754894429_1964327_1376394253_0c49a009419f2781ad5e0704a1ccc2b6_rle_crop_3910817888_0.png resize: (991, 1454) 1376400587 -0.5323163413293701 treat image : temp/1754894429_1964327_1376394253_0c49a009419f2781ad5e0704a1ccc2b6_rle_crop_3910817887_0.png resize: (144, 179) 1376400588 -2.133902778916597 treat image : temp/1754894429_1964327_1376394253_0c49a009419f2781ad5e0704a1ccc2b6_rle_crop_3910817890_0.png resize: (471, 343) 1376400589 -0.5493196043964362 treat image : temp/1754894429_1964327_1376394253_0c49a009419f2781ad5e0704a1ccc2b6_rle_crop_3910817889_0.png resize: (86, 82) 1376400590 1.1560949166607737 treat image : temp/1754894429_1964327_1376394304_51c8f2f0ac8434b9161b3d8f00cb8bde_rle_crop_3910817863_0.png resize: (124, 117) 1376400593 -1.362308674975456 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 56 time used for this insertion : 0.014854907989501953 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 56 time used for this insertion : 0.09490180015563965 save missing photos in datou_result : time spend for datou_step_exec : 16.42439341545105 time spend to save output : 0.1153872013092041 total time spend for step 6 : 16.539780616760254 step7:brightness Mon Aug 11 08:42:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1754894429_1964327_1376394333_adb12429fe61ed91c45138fd452e798b.jpg treat image : temp/1754894429_1964327_1376394330_f7f1dfa1cf4687c48a060ba70ebdaab0.jpg treat image : temp/1754894429_1964327_1376394327_5817ce15f9f853f7457f21d76e8011a4.jpg treat image : temp/1754894429_1964327_1376394325_d023545f5d488b6edd76fd9fd3925f8d.jpg treat image : temp/1754894429_1964327_1376394306_249032193d7b38e1900507dea8155161.jpg treat image : temp/1754894429_1964327_1376394304_51c8f2f0ac8434b9161b3d8f00cb8bde.jpg treat image : temp/1754894429_1964327_1376394301_3dea6e8919d18c72d7a78a8f6cc97515.jpg treat image : temp/1754894429_1964327_1376394297_7173e552cbfa7ba478abe879349fff6b.jpg treat image : temp/1754894429_1964327_1376394292_57f47de61c4b9cd1afb0f939e23a0754.jpg treat image : temp/1754894429_1964327_1376394289_7650914cbc7e44b13000eb799f22bff6.jpg treat image : temp/1754894429_1964327_1376394265_3929f859b20f0fbc54baba4bb9c5746c.jpg treat image : temp/1754894429_1964327_1376394262_277dcc8ec2a8f0b2b94dbb209540e4b9.jpg treat image : temp/1754894429_1964327_1376394260_6703a01f781df4d2217368af0babf4e6.jpg treat image : temp/1754894429_1964327_1376394256_7d879e5217e117e6f37b3e66916c3061.jpg treat image : temp/1754894429_1964327_1376394253_0c49a009419f2781ad5e0704a1ccc2b6.jpg treat image : temp/1754894429_1964327_1376394333_adb12429fe61ed91c45138fd452e798b_rle_crop_3910817854_0.png treat image : temp/1754894429_1964327_1376394333_adb12429fe61ed91c45138fd452e798b_rle_crop_3910817853_0.png treat image : temp/1754894429_1964327_1376394330_f7f1dfa1cf4687c48a060ba70ebdaab0_rle_crop_3910817855_0.png treat image : temp/1754894429_1964327_1376394325_d023545f5d488b6edd76fd9fd3925f8d_rle_crop_3910817857_0.png treat image : temp/1754894429_1964327_1376394306_249032193d7b38e1900507dea8155161_rle_crop_3910817859_0.png treat image : temp/1754894429_1964327_1376394306_249032193d7b38e1900507dea8155161_rle_crop_3910817860_0.png treat image : temp/1754894429_1964327_1376394304_51c8f2f0ac8434b9161b3d8f00cb8bde_rle_crop_3910817862_0.png treat image : temp/1754894429_1964327_1376394301_3dea6e8919d18c72d7a78a8f6cc97515_rle_crop_3910817864_0.png treat image : temp/1754894429_1964327_1376394292_57f47de61c4b9cd1afb0f939e23a0754_rle_crop_3910817868_0.png treat image : temp/1754894429_1964327_1376394289_7650914cbc7e44b13000eb799f22bff6_rle_crop_3910817871_0.png treat image : temp/1754894429_1964327_1376394265_3929f859b20f0fbc54baba4bb9c5746c_rle_crop_3910817872_0.png treat image : temp/1754894429_1964327_1376394265_3929f859b20f0fbc54baba4bb9c5746c_rle_crop_3910817873_0.png treat image : temp/1754894429_1964327_1376394262_277dcc8ec2a8f0b2b94dbb209540e4b9_rle_crop_3910817877_0.png treat image : temp/1754894429_1964327_1376394262_277dcc8ec2a8f0b2b94dbb209540e4b9_rle_crop_3910817876_0.png treat image : temp/1754894429_1964327_1376394260_6703a01f781df4d2217368af0babf4e6_rle_crop_3910817879_0.png treat image : temp/1754894429_1964327_1376394256_7d879e5217e117e6f37b3e66916c3061_rle_crop_3910817884_0.png treat image : temp/1754894429_1964327_1376394256_7d879e5217e117e6f37b3e66916c3061_rle_crop_3910817883_0.png treat image : temp/1754894429_1964327_1376394256_7d879e5217e117e6f37b3e66916c3061_rle_crop_3910817885_0.png treat image : temp/1754894429_1964327_1376394256_7d879e5217e117e6f37b3e66916c3061_rle_crop_3910817882_0.png treat image : temp/1754894429_1964327_1376394253_0c49a009419f2781ad5e0704a1ccc2b6_rle_crop_3910817891_0.png treat image : temp/1754894429_1964327_1376394325_d023545f5d488b6edd76fd9fd3925f8d_rle_crop_3910817858_0.png treat image : temp/1754894429_1964327_1376394256_7d879e5217e117e6f37b3e66916c3061_rle_crop_3910817886_0.png treat image : temp/1754894429_1964327_1376394256_7d879e5217e117e6f37b3e66916c3061_rle_crop_3910817881_0.png treat image : temp/1754894429_1964327_1376394333_adb12429fe61ed91c45138fd452e798b_rle_crop_3910817852_0.png treat image : temp/1754894429_1964327_1376394333_adb12429fe61ed91c45138fd452e798b_rle_crop_3910817851_0.png treat image : temp/1754894429_1964327_1376394327_5817ce15f9f853f7457f21d76e8011a4_rle_crop_3910817856_0.png treat image : temp/1754894429_1964327_1376394306_249032193d7b38e1900507dea8155161_rle_crop_3910817861_0.png treat image : temp/1754894429_1964327_1376394297_7173e552cbfa7ba478abe879349fff6b_rle_crop_3910817866_0.png treat image : temp/1754894429_1964327_1376394297_7173e552cbfa7ba478abe879349fff6b_rle_crop_3910817867_0.png treat image : temp/1754894429_1964327_1376394297_7173e552cbfa7ba478abe879349fff6b_rle_crop_3910817865_0.png treat image : temp/1754894429_1964327_1376394292_57f47de61c4b9cd1afb0f939e23a0754_rle_crop_3910817870_0.png treat image : temp/1754894429_1964327_1376394292_57f47de61c4b9cd1afb0f939e23a0754_rle_crop_3910817869_0.png treat image : temp/1754894429_1964327_1376394265_3929f859b20f0fbc54baba4bb9c5746c_rle_crop_3910817875_0.png treat image : temp/1754894429_1964327_1376394265_3929f859b20f0fbc54baba4bb9c5746c_rle_crop_3910817874_0.png treat image : temp/1754894429_1964327_1376394262_277dcc8ec2a8f0b2b94dbb209540e4b9_rle_crop_3910817878_0.png treat image : temp/1754894429_1964327_1376394260_6703a01f781df4d2217368af0babf4e6_rle_crop_3910817880_0.png treat image : temp/1754894429_1964327_1376394253_0c49a009419f2781ad5e0704a1ccc2b6_rle_crop_3910817888_0.png treat image : temp/1754894429_1964327_1376394253_0c49a009419f2781ad5e0704a1ccc2b6_rle_crop_3910817887_0.png treat image : temp/1754894429_1964327_1376394253_0c49a009419f2781ad5e0704a1ccc2b6_rle_crop_3910817890_0.png treat image : temp/1754894429_1964327_1376394253_0c49a009419f2781ad5e0704a1ccc2b6_rle_crop_3910817889_0.png treat image : temp/1754894429_1964327_1376394304_51c8f2f0ac8434b9161b3d8f00cb8bde_rle_crop_3910817863_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 56 time used for this insertion : 0.020312786102294922 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 56 time used for this insertion : 0.024266719818115234 save missing photos in datou_result : time spend for datou_step_exec : 4.659317493438721 time spend to save output : 0.04903769493103027 total time spend for step 7 : 4.708355188369751 step8:velours_tree Mon Aug 11 08:42:34 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.08369827270507812 time spend to save output : 4.1484832763671875e-05 total time spend for step 8 : 0.0837397575378418 step9:send_mail_cod Mon Aug 11 08:42:34 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P25867659_11-08-2025_08_42_34.pdf 25867949 imagette258679491754894554 25867950 change filename to text .change filename to text .change filename to text .imagette258679501754894554 25867951 imagette258679511754894554 25867952 imagette258679521754894554 25867953 imagette258679531754894554 25867954 change filename to text .imagette258679541754894554 25867955 imagette258679551754894554 25867956 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette258679561754894554 25867958 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette258679581754894555 25867959 imagette258679591754894557 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=25867659 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/25867949,25867950,25867951,25867952,25867953,25867954,25867955,25867956,25867957,25867958,25867959?tags=mal_croppe,carton,pet_fonce,background,metal,autre,pehd,papier,environnement,pet_clair,flou args[1376394333] : ((1376394333, -0.06997299114849541, 492688767), (1376394333, 0.7032982482626118, 2107752395), '0.2745930748456791') We are sending mail with results at report@fotonower.com args[1376394330] : ((1376394330, 0.8121591041532854, 492688767), (1376394330, 0.8460501232051013, 2107752395), '0.2745930748456791') We are sending mail with results at report@fotonower.com args[1376394327] : ((1376394327, 0.7913162251184331, 492688767), (1376394327, 0.7087813366347724, 2107752395), '0.2745930748456791') We are sending mail with results at report@fotonower.com args[1376394325] : ((1376394325, 1.0179652203596479, 492688767), (1376394325, 0.8433924811132523, 2107752395), '0.2745930748456791') We are sending mail with results at report@fotonower.com args[1376394306] : ((1376394306, 1.3536295339085098, 492688767), (1376394306, 0.6340901679662765, 2107752395), '0.2745930748456791') We are sending mail with results at report@fotonower.com args[1376394304] : ((1376394304, 5.1140010446564474, 492688767), (1376394304, 0.6207344829195561, 2107752395), '0.2745930748456791') We are sending mail with results at report@fotonower.com args[1376394301] : ((1376394301, -0.10098682489087217, 492688767), (1376394301, 0.7988151875122296, 2107752395), '0.2745930748456791') We are sending mail with results at report@fotonower.com args[1376394297] : ((1376394297, 5.0310345206761, 492688767), (1376394297, 0.8420045428207693, 2107752395), '0.2745930748456791') We are sending mail with results at report@fotonower.com args[1376394292] : ((1376394292, 2.01177056274147, 492688767), (1376394292, 0.7722659669675869, 2107752395), '0.2745930748456791') We are sending mail with results at report@fotonower.com args[1376394289] : ((1376394289, 1.1665181516534016, 492688767), (1376394289, 1.0283721936363226, 2107752395), '0.2745930748456791') We are sending mail with results at report@fotonower.com args[1376394265] : ((1376394265, 3.275973894165184, 492688767), (1376394265, 0.6904705019980667, 2107752395), '0.2745930748456791') We are sending mail with results at report@fotonower.com args[1376394262] : ((1376394262, 1.576227242552276, 492688767), (1376394262, 1.123154912227517, 2107752395), '0.2745930748456791') We are sending mail with results at report@fotonower.com args[1376394260] : ((1376394260, 3.231447724769417, 492688767), (1376394260, 0.5983446294769691, 2107752395), '0.2745930748456791') We are sending mail with results at report@fotonower.com args[1376394256] : ((1376394256, 1.1668438586351026, 492688767), (1376394256, 1.0008675010018553, 2107752395), '0.2745930748456791') We are sending mail with results at report@fotonower.com args[1376394253] : ((1376394253, 2.513552909865967, 492688767), (1376394253, 0.7829421808430086, 2107752395), '0.2745930748456791') We are sending mail with results at report@fotonower.com refus_total : 0.2745930748456791 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=25867659 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25867659_11-08-2025_08_42_34.pdf results_Auto_P25867659_11-08-2025_08_42_34.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25867659_11-08-2025_08_42_34.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','25867659','results_Auto_P25867659_11-08-2025_08_42_34.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25867659_11-08-2025_08_42_34.pdf','pdf','','0.88','0.2745930748456791') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/25867659

https://www.fotonower.com/image?json=false&list_photos_id=1376394333
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1376394330
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1376394327
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1376394325
La photo est trop floue, merci de reprendre une photo.(avec le score = 1.0179652203596479)
https://www.fotonower.com/image?json=false&list_photos_id=1376394306
La photo est trop floue, merci de reprendre une photo.(avec le score = 1.3536295339085098)
https://www.fotonower.com/image?json=false&list_photos_id=1376394304
La photo est trop floue, merci de reprendre une photo.(avec le score = 5.1140010446564474)
https://www.fotonower.com/image?json=false&list_photos_id=1376394301
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1376394297
La photo est trop floue, merci de reprendre une photo.(avec le score = 5.0310345206761)
https://www.fotonower.com/image?json=false&list_photos_id=1376394292
La photo est trop floue, merci de reprendre une photo.(avec le score = 2.01177056274147)
https://www.fotonower.com/image?json=false&list_photos_id=1376394289
La photo est trop floue, merci de reprendre une photo.(avec le score = 1.1665181516534016)
https://www.fotonower.com/image?json=false&list_photos_id=1376394265
La photo est trop floue, merci de reprendre une photo.(avec le score = 3.275973894165184)
https://www.fotonower.com/image?json=false&list_photos_id=1376394262
La photo est trop floue, merci de reprendre une photo.(avec le score = 1.576227242552276)
https://www.fotonower.com/image?json=false&list_photos_id=1376394260
La photo est trop floue, merci de reprendre une photo.(avec le score = 3.231447724769417)
https://www.fotonower.com/image?json=false&list_photos_id=1376394256
La photo est trop floue, merci de reprendre une photo.(avec le score = 1.1668438586351026)
https://www.fotonower.com/image?json=false&list_photos_id=1376394253
La photo est trop floue, merci de reprendre une photo.(avec le score = 2.513552909865967)

Dans ces conditions,le taux de refus est: 27.46%
Veuillez trouver les photos des contaminants.

exemples de contaminants: carton: https://www.fotonower.com/view/25867950?limit=200
exemples de contaminants: autre: https://www.fotonower.com/view/25867954?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/25867956?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/25867958?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25867659_11-08-2025_08_42_34.pdf.

Lien vers velours :https://www.fotonower.com/velours/25867949,25867950,25867951,25867952,25867953,25867954,25867955,25867956,25867957,25867958,25867959?tags=mal_croppe,carton,pet_fonce,background,metal,autre,pehd,papier,environnement,pet_clair,flou.


L'équipe Fotonower 202 b'' Server: nginx Date: Mon, 11 Aug 2025 06:42:39 GMT Content-Length: 0 Connection: close X-Message-Id: SByzQSXkTQ6S06dM7xVgfg Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1376394333, 1376394330, 1376394327, 1376394325, 1376394306, 1376394304, 1376394301, 1376394297, 1376394292, 1376394289, 1376394265, 1376394262, 1376394260, 1376394256, 1376394253] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394333', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394330', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394327', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394325', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394306', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394304', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394301', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394297', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394292', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394289', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394265', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394262', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394260', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394256', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394253', None, None, None, None, None, '3496192') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 15 time used for this insertion : 0.04473114013671875 save_final save missing photos in datou_result : time spend for datou_step_exec : 5.257878303527832 time spend to save output : 0.04494953155517578 total time spend for step 9 : 5.302827835083008 step10:split_time_score Mon Aug 11 08:42:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('07', 15),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 11082025 25867659 Nombre de photos uploadées : 15 / 23040 (0%) 11082025 25867659 Nombre de photos taguées (types de déchets): 0 / 15 (0%) 11082025 25867659 Nombre de photos taguées (volume) : 0 / 15 (0%) elapsed_time : load_data_split_time_score 1.1920928955078125e-06 elapsed_time : order_list_meta_photo_and_scores 5.0067901611328125e-06 ??????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0007224082946777344 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.20356082916259766 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25867656 order by id desc limit 1 Qualite : 0.2745930748456791 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25867659_11-08-2025_08_42_34.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25867659 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=25867659 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'11082025': {'nb_upload': 15, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1376394333, 1376394330, 1376394327, 1376394325, 1376394306, 1376394304, 1376394301, 1376394297, 1376394292, 1376394289, 1376394265, 1376394262, 1376394260, 1376394256, 1376394253] Looping around the photos to save general results len do output : 1 /25867659Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394333', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394330', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394327', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394325', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394306', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394304', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394301', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394297', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394292', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394289', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394265', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394262', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394260', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394256', None, None, None, None, None, '3496192') ('3318', None, None, None, None, None, None, None, '3496192') ('3318', '25867659', '1376394253', None, None, None, None, None, '3496192') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 16 time used for this insertion : 0.015819311141967773 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.425100564956665 time spend to save output : 0.016073942184448242 total time spend for step 10 : 1.4411745071411133 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 15 set_done_treatment 83.59user 28.93system 2:16.10elapsed 82%CPU (0avgtext+0avgdata 2887220maxresident)k 501776inputs+57560outputs (1major+2108285minor)pagefaults 0swaps