python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2145937 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['3754488'] with mtr_portfolio_ids : ['27071629'] and first list_photo_ids : [] new path : /proc/2145937/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 12 ; length of list_pids : 12 ; length of list_args : 12 time to download the photos : 2.4182960987091064 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Sun Sep 21 08:50:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10588 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-09-21 08:50:34.317773: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-09-21 08:50:34.344598: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-09-21 08:50:34.346219: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f5440000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-09-21 08:50:34.346268: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-09-21 08:50:34.348884: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-09-21 08:50:34.524890: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x25ed8d00 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-09-21 08:50:34.524946: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-09-21 08:50:34.525869: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-21 08:50:34.526182: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-21 08:50:34.531788: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-21 08:50:34.533978: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-21 08:50:34.534315: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-21 08:50:34.536833: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-21 08:50:34.537995: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-21 08:50:34.543352: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-21 08:50:34.544868: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-21 08:50:34.544949: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-21 08:50:34.545716: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-21 08:50:34.545733: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-21 08:50:34.545742: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-21 08:50:34.547071: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9807 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-09-21 08:50:34.793702: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-21 08:50:34.793822: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-21 08:50:34.793839: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-21 08:50:34.793853: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-21 08:50:34.793867: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-21 08:50:34.793880: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-21 08:50:34.793894: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-21 08:50:34.793908: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-21 08:50:34.795086: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-21 08:50:34.796358: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-21 08:50:34.796392: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-21 08:50:34.796407: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-21 08:50:34.796420: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-21 08:50:34.796432: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-21 08:50:34.796445: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-21 08:50:34.796458: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-21 08:50:34.796471: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-21 08:50:34.797631: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-21 08:50:34.797666: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-21 08:50:34.797674: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-21 08:50:34.797681: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-21 08:50:34.798894: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9807 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-09-21 08:50:45.188114: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-21 08:50:45.358219: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 12 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 24 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 23 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 18 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 18 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 21 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 23 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 11 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 14 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 17 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 17 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 21 Detection mask done ! Trying to reset tf kernel 2146423 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 415 tf kernel not reseted sub process len(results) : 12 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 12 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5704 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0013072490692138672 nb_pixel_total : 35554 time to create 1 rle with old method : 0.04573798179626465 length of segment : 350 time for calcul the mask position with numpy : 0.0004563331604003906 nb_pixel_total : 26185 time to create 1 rle with old method : 0.034790992736816406 length of segment : 131 time for calcul the mask position with numpy : 0.0003066062927246094 nb_pixel_total : 17359 time to create 1 rle with old method : 0.021279573440551758 length of segment : 116 time for calcul the mask position with numpy : 0.0015323162078857422 nb_pixel_total : 61018 time to create 1 rle with old method : 0.07525753974914551 length of segment : 317 time for calcul the mask position with numpy : 0.0005357265472412109 nb_pixel_total : 32591 time to create 1 rle with old method : 0.03984951972961426 length of segment : 222 time for calcul the mask position with numpy : 0.022541046142578125 nb_pixel_total : 389774 time to create 1 rle with new method : 0.029927968978881836 length of segment : 769 time for calcul the mask position with numpy : 0.0012919902801513672 nb_pixel_total : 85568 time to create 1 rle with old method : 0.10153436660766602 length of segment : 358 time for calcul the mask position with numpy : 0.0011456012725830078 nb_pixel_total : 55010 time to create 1 rle with old method : 0.12856483459472656 length of segment : 257 time for calcul the mask position with numpy : 0.0007174015045166016 nb_pixel_total : 37327 time to create 1 rle with old method : 0.044500112533569336 length of segment : 179 time for calcul the mask position with numpy : 0.0007066726684570312 nb_pixel_total : 41967 time to create 1 rle with old method : 0.05475449562072754 length of segment : 312 time for calcul the mask position with numpy : 0.0010318756103515625 nb_pixel_total : 39333 time to create 1 rle with old method : 0.053469181060791016 length of segment : 294 time for calcul the mask position with numpy : 0.00027632713317871094 nb_pixel_total : 9878 time to create 1 rle with old method : 0.012230396270751953 length of segment : 135 time for calcul the mask position with numpy : 0.0003714561462402344 nb_pixel_total : 18639 time to create 1 rle with old method : 0.022144079208374023 length of segment : 173 time for calcul the mask position with numpy : 0.0013685226440429688 nb_pixel_total : 80348 time to create 1 rle with old method : 0.09417366981506348 length of segment : 300 time for calcul the mask position with numpy : 0.0015439987182617188 nb_pixel_total : 94403 time to create 1 rle with old method : 0.11765575408935547 length of segment : 394 time for calcul the mask position with numpy : 0.0006945133209228516 nb_pixel_total : 28665 time to create 1 rle with old method : 0.034619808197021484 length of segment : 221 time for calcul the mask position with numpy : 0.0011146068572998047 nb_pixel_total : 67912 time to create 1 rle with old method : 0.08033943176269531 length of segment : 307 time for calcul the mask position with numpy : 0.00039196014404296875 nb_pixel_total : 18786 time to create 1 rle with old method : 0.02214670181274414 length of segment : 190 time for calcul the mask position with numpy : 0.0008952617645263672 nb_pixel_total : 61629 time to create 1 rle with old method : 0.07340121269226074 length of segment : 300 time for calcul the mask position with numpy : 0.00024962425231933594 nb_pixel_total : 12206 time to create 1 rle with old method : 0.014502286911010742 length of segment : 146 time for calcul the mask position with numpy : 0.0014200210571289062 nb_pixel_total : 97718 time to create 1 rle with old method : 0.11455249786376953 length of segment : 404 time for calcul the mask position with numpy : 0.0003917217254638672 nb_pixel_total : 24816 time to create 1 rle with old method : 0.029263734817504883 length of segment : 171 time for calcul the mask position with numpy : 0.0004687309265136719 nb_pixel_total : 27888 time to create 1 rle with old method : 0.03301119804382324 length of segment : 228 time for calcul the mask position with numpy : 0.0005877017974853516 nb_pixel_total : 39227 time to create 1 rle with old method : 0.05723261833190918 length of segment : 254 time for calcul the mask position with numpy : 0.00019431114196777344 nb_pixel_total : 8833 time to create 1 rle with old method : 0.01063394546508789 length of segment : 127 time for calcul the mask position with numpy : 0.0011034011840820312 nb_pixel_total : 71919 time to create 1 rle with old method : 0.08901381492614746 length of segment : 236 time for calcul the mask position with numpy : 0.0005867481231689453 nb_pixel_total : 40093 time to create 1 rle with old method : 0.04709434509277344 length of segment : 221 time for calcul the mask position with numpy : 0.0006313323974609375 nb_pixel_total : 37338 time to create 1 rle with old method : 0.043770551681518555 length of segment : 455 time for calcul the mask position with numpy : 0.0002779960632324219 nb_pixel_total : 10269 time to create 1 rle with old method : 0.01770925521850586 length of segment : 130 time for calcul the mask position with numpy : 0.0013935565948486328 nb_pixel_total : 84432 time to create 1 rle with old method : 0.09830331802368164 length of segment : 372 time for calcul the mask position with numpy : 0.0004227161407470703 nb_pixel_total : 23242 time to create 1 rle with old method : 0.027619600296020508 length of segment : 168 time for calcul the mask position with numpy : 0.0013391971588134766 nb_pixel_total : 81581 time to create 1 rle with old method : 0.10967421531677246 length of segment : 318 time for calcul the mask position with numpy : 0.0005497932434082031 nb_pixel_total : 13876 time to create 1 rle with old method : 0.016518592834472656 length of segment : 131 time for calcul the mask position with numpy : 0.0016167163848876953 nb_pixel_total : 64360 time to create 1 rle with old method : 0.0752255916595459 length of segment : 271 time for calcul the mask position with numpy : 0.002208232879638672 nb_pixel_total : 82419 time to create 1 rle with old method : 0.09000539779663086 length of segment : 289 time for calcul the mask position with numpy : 0.0011744499206542969 nb_pixel_total : 64157 time to create 1 rle with old method : 0.07224845886230469 length of segment : 419 time for calcul the mask position with numpy : 0.0005443096160888672 nb_pixel_total : 22477 time to create 1 rle with old method : 0.025098085403442383 length of segment : 207 time for calcul the mask position with numpy : 0.00031757354736328125 nb_pixel_total : 18081 time to create 1 rle with old method : 0.020686626434326172 length of segment : 125 time for calcul the mask position with numpy : 0.0002231597900390625 nb_pixel_total : 11372 time to create 1 rle with old method : 0.012906074523925781 length of segment : 132 time for calcul the mask position with numpy : 0.0013685226440429688 nb_pixel_total : 95384 time to create 1 rle with old method : 0.10515522956848145 length of segment : 326 time for calcul the mask position with numpy : 0.008792638778686523 nb_pixel_total : 112156 time to create 1 rle with old method : 0.12615084648132324 length of segment : 373 time for calcul the mask position with numpy : 0.0011296272277832031 nb_pixel_total : 41844 time to create 1 rle with old method : 0.04640388488769531 length of segment : 236 time for calcul the mask position with numpy : 0.00034046173095703125 nb_pixel_total : 8647 time to create 1 rle with old method : 0.009918451309204102 length of segment : 99 time for calcul the mask position with numpy : 0.0007865428924560547 nb_pixel_total : 30805 time to create 1 rle with old method : 0.034827232360839844 length of segment : 167 time for calcul the mask position with numpy : 0.0012199878692626953 nb_pixel_total : 45415 time to create 1 rle with old method : 0.05219745635986328 length of segment : 248 time for calcul the mask position with numpy : 0.0013692378997802734 nb_pixel_total : 47941 time to create 1 rle with old method : 0.05403637886047363 length of segment : 256 time for calcul the mask position with numpy : 0.0010807514190673828 nb_pixel_total : 30978 time to create 1 rle with old method : 0.03432178497314453 length of segment : 248 time for calcul the mask position with numpy : 0.0017218589782714844 nb_pixel_total : 46393 time to create 1 rle with old method : 0.05182290077209473 length of segment : 326 time for calcul the mask position with numpy : 0.0008828639984130859 nb_pixel_total : 23593 time to create 1 rle with old method : 0.025756120681762695 length of segment : 213 time for calcul the mask position with numpy : 0.0016024112701416016 nb_pixel_total : 62022 time to create 1 rle with old method : 0.07343459129333496 length of segment : 360 time for calcul the mask position with numpy : 0.0017385482788085938 nb_pixel_total : 66097 time to create 1 rle with old method : 0.07200860977172852 length of segment : 363 time for calcul the mask position with numpy : 0.0008823871612548828 nb_pixel_total : 36398 time to create 1 rle with old method : 0.04016733169555664 length of segment : 234 time for calcul the mask position with numpy : 0.002391815185546875 nb_pixel_total : 83267 time to create 1 rle with old method : 0.10539674758911133 length of segment : 350 time for calcul the mask position with numpy : 0.0005686283111572266 nb_pixel_total : 12208 time to create 1 rle with old method : 0.015559196472167969 length of segment : 77 time for calcul the mask position with numpy : 0.0006425380706787109 nb_pixel_total : 19954 time to create 1 rle with old method : 0.024569034576416016 length of segment : 151 time for calcul the mask position with numpy : 0.0020875930786132812 nb_pixel_total : 57281 time to create 1 rle with old method : 0.06980061531066895 length of segment : 239 time for calcul the mask position with numpy : 0.0013625621795654297 nb_pixel_total : 49419 time to create 1 rle with old method : 0.05987381935119629 length of segment : 217 time for calcul the mask position with numpy : 0.0025756359100341797 nb_pixel_total : 91735 time to create 1 rle with old method : 0.13248586654663086 length of segment : 257 time for calcul the mask position with numpy : 0.0010650157928466797 nb_pixel_total : 46240 time to create 1 rle with old method : 0.05800580978393555 length of segment : 187 time for calcul the mask position with numpy : 0.001674652099609375 nb_pixel_total : 37590 time to create 1 rle with old method : 0.04516267776489258 length of segment : 575 time for calcul the mask position with numpy : 0.0006439685821533203 nb_pixel_total : 17164 time to create 1 rle with old method : 0.02107405662536621 length of segment : 135 time for calcul the mask position with numpy : 0.0010585784912109375 nb_pixel_total : 39918 time to create 1 rle with old method : 0.06302809715270996 length of segment : 239 time for calcul the mask position with numpy : 0.0025413036346435547 nb_pixel_total : 88362 time to create 1 rle with old method : 0.10346388816833496 length of segment : 298 time for calcul the mask position with numpy : 0.003809690475463867 nb_pixel_total : 167568 time to create 1 rle with new method : 0.007527351379394531 length of segment : 555 time for calcul the mask position with numpy : 0.0013773441314697266 nb_pixel_total : 51073 time to create 1 rle with old method : 0.05962491035461426 length of segment : 234 time for calcul the mask position with numpy : 0.0005633831024169922 nb_pixel_total : 16530 time to create 1 rle with old method : 0.019835233688354492 length of segment : 131 time for calcul the mask position with numpy : 0.004998922348022461 nb_pixel_total : 177113 time to create 1 rle with new method : 0.011637687683105469 length of segment : 601 time for calcul the mask position with numpy : 0.0009250640869140625 nb_pixel_total : 16266 time to create 1 rle with old method : 0.01953721046447754 length of segment : 437 time for calcul the mask position with numpy : 0.0017845630645751953 nb_pixel_total : 74037 time to create 1 rle with old method : 0.08668661117553711 length of segment : 282 time for calcul the mask position with numpy : 0.0009555816650390625 nb_pixel_total : 26385 time to create 1 rle with old method : 0.03113842010498047 length of segment : 223 time for calcul the mask position with numpy : 0.004393577575683594 nb_pixel_total : 154427 time to create 1 rle with new method : 0.010439157485961914 length of segment : 364 time for calcul the mask position with numpy : 0.0007774829864501953 nb_pixel_total : 28665 time to create 1 rle with old method : 0.03468012809753418 length of segment : 212 time for calcul the mask position with numpy : 0.0024881362915039062 nb_pixel_total : 84902 time to create 1 rle with old method : 0.09810781478881836 length of segment : 381 time for calcul the mask position with numpy : 0.0015497207641601562 nb_pixel_total : 34440 time to create 1 rle with old method : 0.041185617446899414 length of segment : 309 time for calcul the mask position with numpy : 0.0005393028259277344 nb_pixel_total : 15324 time to create 1 rle with old method : 0.018381595611572266 length of segment : 120 time for calcul the mask position with numpy : 0.0024938583374023438 nb_pixel_total : 67408 time to create 1 rle with old method : 0.0795586109161377 length of segment : 436 time for calcul the mask position with numpy : 0.0015616416931152344 nb_pixel_total : 58891 time to create 1 rle with old method : 0.07185053825378418 length of segment : 306 time for calcul the mask position with numpy : 0.001344919204711914 nb_pixel_total : 42880 time to create 1 rle with old method : 0.050844430923461914 length of segment : 281 time spent for convertir_results : 8.527799367904663 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 78 chid ids of type : 3594 Number RLEs to save : 21175 save missing photos in datou_result : time spend for datou_step_exec : 59.56857371330261 time spend to save output : 1.6673901081085205 total time spend for step 1 : 61.23596382141113 step2:crop_condition Sun Sep 21 08:51:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 12 ! batch 1 Loaded 78 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 30 About to insert : list_path_to_insert length 30 new photo from crops ! About to upload 30 photos upload in portfolio : 3736932 init cache_photo without model_param we have 30 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758437502_2145937 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 30 photos in the portfolio 3736932 time of upload the photos Elapsed time : 7.539164781570435 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 13 About to insert : list_path_to_insert length 13 new photo from crops ! About to upload 13 photos upload in portfolio : 3736932 init cache_photo without model_param we have 13 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758437515_2145937 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 13 photos in the portfolio 3736932 time of upload the photos Elapsed time : 3.862661838531494 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 3736932 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758437522_2145937 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 4 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.8421871662139893 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 25 About to insert : list_path_to_insert length 25 new photo from crops ! About to upload 25 photos upload in portfolio : 3736932 init cache_photo without model_param we have 25 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758437532_2145937 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 25 photos in the portfolio 3736932 time of upload the photos Elapsed time : 8.792777061462402 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 3736932 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758437542_2145937 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 5 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.5584092140197754 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758437546_2145937 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.6147956848144531 we have finished the crop for the class : pet_fonce delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1385303709, 1385303675, 1385303674, 1385303671, 1385303670, 1385303669, 1385303668, 1385303634, 1385303632, 1385303630, 1385303628, 1385303627] Looping around the photos to save general results len do output : 78 /1385321200Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321202Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321203Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321204Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321205Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321206Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321207Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321208Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321209Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321210Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321211Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321212Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321213Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321214Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321215Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321216Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321217Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321218Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321219Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321220Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321221Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321222Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321223Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321224Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321225Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321226Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321227Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321228Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321229Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321230Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321234Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321235Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321236Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321237Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321238Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321239Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321240Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321241Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321242Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321243Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321244Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321245Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321246Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321248Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321249Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321250Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321251Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321268Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321269Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321270Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321271Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321272Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321273Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321274Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321275Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321276Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321277Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321278Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321279Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321280Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321281Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321282Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321283Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321284Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321285Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321286Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321287Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321288Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321289Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321290Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321291Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321292Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321295Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321296Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321297Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321298Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321299Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1385321301Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303709', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303675', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303674', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303671', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303670', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303669', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303668', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303634', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303632', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303630', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303628', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303627', None, None, None, None, None, '3754488') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 246 time used for this insertion : 0.5291028022766113 save_final save missing photos in datou_result : time spend for datou_step_exec : 53.541486978530884 time spend to save output : 0.5314955711364746 total time spend for step 2 : 54.07298254966736 step3:rle_unique_nms_with_priority Sun Sep 21 08:52:26 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 78 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 7 nb_hashtags : 4 time to prepare the origin masks : 3.7772490978240967 time for calcul the mask position with numpy : 0.64385986328125 nb_pixel_total : 7646351 time to create 1 rle with new method : 0.780205249786377 time for calcul the mask position with numpy : 0.024782180786132812 nb_pixel_total : 85568 time to create 1 rle with old method : 0.09732365608215332 time for calcul the mask position with numpy : 0.03624773025512695 nb_pixel_total : 389774 time to create 1 rle with new method : 0.7532703876495361 time for calcul the mask position with numpy : 0.03043818473815918 nb_pixel_total : 32591 time to create 1 rle with old method : 0.03898811340332031 time for calcul the mask position with numpy : 0.027247190475463867 nb_pixel_total : 61018 time to create 1 rle with old method : 0.07028579711914062 time for calcul the mask position with numpy : 0.024731159210205078 nb_pixel_total : 17359 time to create 1 rle with old method : 0.020084619522094727 time for calcul the mask position with numpy : 0.026253223419189453 nb_pixel_total : 26185 time to create 1 rle with old method : 0.029549598693847656 time for calcul the mask position with numpy : 0.026175975799560547 nb_pixel_total : 35554 time to create 1 rle with old method : 0.041588783264160156 create new chi : 2.7370662689208984 time to delete rle : 0.02031421661376953 batch 1 Loaded 15 chid ids of type : 3594 +++++++++Number RLEs to save : 6686 TO DO : save crop sub photo not yet done ! save time : 1.0470736026763916 nb_obj : 11 nb_hashtags : 5 time to prepare the origin masks : 5.004754304885864 time for calcul the mask position with numpy : 0.5260369777679443 nb_pixel_total : 7802132 time to create 1 rle with new method : 0.919086217880249 time for calcul the mask position with numpy : 0.02632927894592285 nb_pixel_total : 18786 time to create 1 rle with old method : 0.02861332893371582 time for calcul the mask position with numpy : 0.029248714447021484 nb_pixel_total : 67912 time to create 1 rle with old method : 0.08683419227600098 time for calcul the mask position with numpy : 0.027948856353759766 nb_pixel_total : 28665 time to create 1 rle with old method : 0.03306412696838379 time for calcul the mask position with numpy : 0.028165578842163086 nb_pixel_total : 94403 time to create 1 rle with old method : 0.11445403099060059 time for calcul the mask position with numpy : 0.02551555633544922 nb_pixel_total : 80348 time to create 1 rle with old method : 0.0913078784942627 time for calcul the mask position with numpy : 0.024742841720581055 nb_pixel_total : 18639 time to create 1 rle with old method : 0.02099156379699707 time for calcul the mask position with numpy : 0.024247407913208008 nb_pixel_total : 9878 time to create 1 rle with old method : 0.011158943176269531 time for calcul the mask position with numpy : 0.02438068389892578 nb_pixel_total : 39333 time to create 1 rle with old method : 0.044121503829956055 time for calcul the mask position with numpy : 0.024531841278076172 nb_pixel_total : 41967 time to create 1 rle with old method : 0.04832148551940918 time for calcul the mask position with numpy : 0.024594545364379883 nb_pixel_total : 37327 time to create 1 rle with old method : 0.04233121871948242 time for calcul the mask position with numpy : 0.024623870849609375 nb_pixel_total : 55010 time to create 1 rle with old method : 0.06151533126831055 create new chi : 2.3566179275512695 time to delete rle : 0.0009555816650390625 batch 1 Loaded 23 chid ids of type : 3594 ++++++++++++Number RLEs to save : 7684 TO DO : save crop sub photo not yet done ! save time : 0.8981149196624756 nb_obj : 7 nb_hashtags : 4 time to prepare the origin masks : 3.0925164222717285 time for calcul the mask position with numpy : 0.5987138748168945 nb_pixel_total : 8037870 time to create 1 rle with new method : 0.7078688144683838 time for calcul the mask position with numpy : 0.03696799278259277 nb_pixel_total : 8833 time to create 1 rle with old method : 0.010047435760498047 time for calcul the mask position with numpy : 0.04008889198303223 nb_pixel_total : 23440 time to create 1 rle with old method : 0.027585744857788086 time for calcul the mask position with numpy : 0.024937152862548828 nb_pixel_total : 27888 time to create 1 rle with old method : 0.032121896743774414 time for calcul the mask position with numpy : 0.026755809783935547 nb_pixel_total : 24816 time to create 1 rle with old method : 0.03295326232910156 time for calcul the mask position with numpy : 0.025820493698120117 nb_pixel_total : 97718 time to create 1 rle with old method : 0.11769413948059082 time for calcul the mask position with numpy : 0.026392221450805664 nb_pixel_total : 12206 time to create 1 rle with old method : 0.014480829238891602 time for calcul the mask position with numpy : 0.02650618553161621 nb_pixel_total : 61629 time to create 1 rle with old method : 0.07316708564758301 create new chi : 1.868514060974121 time to delete rle : 0.0007343292236328125 batch 1 Loaded 15 chid ids of type : 3594 +++++++++Number RLEs to save : 5174 TO DO : save crop sub photo not yet done ! save time : 0.6112961769104004 nb_obj : 6 nb_hashtags : 3 time to prepare the origin masks : 2.5036375522613525 time for calcul the mask position with numpy : 0.7178752422332764 nb_pixel_total : 8027107 time to create 1 rle with new method : 0.6697196960449219 time for calcul the mask position with numpy : 0.02350139617919922 nb_pixel_total : 23242 time to create 1 rle with old method : 0.02572345733642578 time for calcul the mask position with numpy : 0.023883342742919922 nb_pixel_total : 84432 time to create 1 rle with old method : 0.09550809860229492 time for calcul the mask position with numpy : 0.023939847946166992 nb_pixel_total : 10269 time to create 1 rle with old method : 0.011574268341064453 time for calcul the mask position with numpy : 0.02419424057006836 nb_pixel_total : 37338 time to create 1 rle with old method : 0.0419621467590332 time for calcul the mask position with numpy : 0.028476715087890625 nb_pixel_total : 40093 time to create 1 rle with old method : 0.04513120651245117 time for calcul the mask position with numpy : 0.0286102294921875 nb_pixel_total : 71919 time to create 1 rle with old method : 0.09104752540588379 create new chi : 1.8921170234680176 time to delete rle : 0.0010192394256591797 batch 1 Loaded 13 chid ids of type : 3594 ++++++++++Number RLEs to save : 5324 TO DO : save crop sub photo not yet done ! save time : 0.5132355690002441 nb_obj : 3 nb_hashtags : 3 time to prepare the origin masks : 1.6684989929199219 time for calcul the mask position with numpy : 0.595475435256958 nb_pixel_total : 8134583 time to create 1 rle with new method : 0.7391743659973145 time for calcul the mask position with numpy : 0.02567577362060547 nb_pixel_total : 64360 time to create 1 rle with old method : 0.07271385192871094 time for calcul the mask position with numpy : 0.024990558624267578 nb_pixel_total : 13876 time to create 1 rle with old method : 0.0159757137298584 time for calcul the mask position with numpy : 0.025679588317871094 nb_pixel_total : 81581 time to create 1 rle with old method : 0.09382820129394531 create new chi : 1.6383614540100098 time to delete rle : 0.0005004405975341797 batch 1 Loaded 7 chid ids of type : 3594 +++Number RLEs to save : 3600 TO DO : save crop sub photo not yet done ! save time : 0.3428642749786377 nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 2.672776937484741 time for calcul the mask position with numpy : 0.7289085388183594 nb_pixel_total : 8107266 time to create 1 rle with new method : 0.8857173919677734 time for calcul the mask position with numpy : 0.02441859245300293 nb_pixel_total : 18081 time to create 1 rle with old method : 0.019900083541870117 time for calcul the mask position with numpy : 0.024070262908935547 nb_pixel_total : 22477 time to create 1 rle with old method : 0.025246858596801758 time for calcul the mask position with numpy : 0.02303457260131836 nb_pixel_total : 64157 time to create 1 rle with old method : 0.07106447219848633 time for calcul the mask position with numpy : 0.02599358558654785 nb_pixel_total : 82419 time to create 1 rle with old method : 0.0899960994720459 create new chi : 1.9603691101074219 time to delete rle : 0.0006875991821289062 batch 1 Loaded 9 chid ids of type : 3594 +++++Number RLEs to save : 4240 TO DO : save crop sub photo not yet done ! save time : 0.4353957176208496 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 1.6981568336486816 time for calcul the mask position with numpy : 0.5549349784851074 nb_pixel_total : 8169777 time to create 1 rle with new method : 0.722628116607666 time for calcul the mask position with numpy : 0.031255483627319336 nb_pixel_total : 17867 time to create 1 rle with old method : 0.021701574325561523 time for calcul the mask position with numpy : 0.024190187454223633 nb_pixel_total : 95384 time to create 1 rle with old method : 0.11047506332397461 time for calcul the mask position with numpy : 0.025792360305786133 nb_pixel_total : 11372 time to create 1 rle with old method : 0.012982845306396484 create new chi : 1.550522804260254 time to delete rle : 0.0005626678466796875 batch 1 Loaded 7 chid ids of type : 3594 +++Number RLEs to save : 3646 TO DO : save crop sub photo not yet done ! save time : 0.3179631233215332 nb_obj : 5 nb_hashtags : 3 time to prepare the origin masks : 2.3685667514801025 time for calcul the mask position with numpy : 0.6724433898925781 nb_pixel_total : 8119748 time to create 1 rle with new method : 0.6775314807891846 time for calcul the mask position with numpy : 0.02400946617126465 nb_pixel_total : 47941 time to create 1 rle with old method : 0.056510210037231445 time for calcul the mask position with numpy : 0.02537250518798828 nb_pixel_total : 45415 time to create 1 rle with old method : 0.051691532135009766 time for calcul the mask position with numpy : 0.03027176856994629 nb_pixel_total : 30805 time to create 1 rle with old method : 0.04102635383605957 time for calcul the mask position with numpy : 0.024649858474731445 nb_pixel_total : 8647 time to create 1 rle with old method : 0.009948015213012695 time for calcul the mask position with numpy : 0.024327754974365234 nb_pixel_total : 41844 time to create 1 rle with old method : 0.04653358459472656 create new chi : 1.7277894020080566 time to delete rle : 0.0005891323089599609 batch 1 Loaded 11 chid ids of type : 3594 ++++++Number RLEs to save : 4172 TO DO : save crop sub photo not yet done ! save time : 0.45128583908081055 nb_obj : 8 nb_hashtags : 3 time to prepare the origin masks : 3.7580902576446533 time for calcul the mask position with numpy : 0.5050358772277832 nb_pixel_total : 7933444 time to create 1 rle with new method : 0.8456130027770996 time for calcul the mask position with numpy : 0.02404165267944336 nb_pixel_total : 12208 time to create 1 rle with old method : 0.014418601989746094 time for calcul the mask position with numpy : 0.024460792541503906 nb_pixel_total : 83267 time to create 1 rle with old method : 0.0954275131225586 time for calcul the mask position with numpy : 0.02501964569091797 nb_pixel_total : 36398 time to create 1 rle with old method : 0.04649925231933594 time for calcul the mask position with numpy : 0.025440216064453125 nb_pixel_total : 66097 time to create 1 rle with old method : 0.08488774299621582 time for calcul the mask position with numpy : 0.03091597557067871 nb_pixel_total : 62022 time to create 1 rle with old method : 0.07467961311340332 time for calcul the mask position with numpy : 0.02438950538635254 nb_pixel_total : 23593 time to create 1 rle with old method : 0.026553869247436523 time for calcul the mask position with numpy : 0.024479389190673828 nb_pixel_total : 46393 time to create 1 rle with old method : 0.051882266998291016 time for calcul the mask position with numpy : 0.02481985092163086 nb_pixel_total : 30978 time to create 1 rle with old method : 0.0338132381439209 create new chi : 2.027064323425293 time to delete rle : 0.0009906291961669922 batch 1 Loaded 17 chid ids of type : 3594 ++++++++Number RLEs to save : 6502 TO DO : save crop sub photo not yet done ! save time : 0.5983905792236328 nb_obj : 7 nb_hashtags : 3 time to prepare the origin masks : 3.525770902633667 time for calcul the mask position with numpy : 0.44571447372436523 nb_pixel_total : 7975017 time to create 1 rle with new method : 0.8013687133789062 time for calcul the mask position with numpy : 0.02433490753173828 nb_pixel_total : 17164 time to create 1 rle with old method : 0.0192413330078125 time for calcul the mask position with numpy : 0.02409672737121582 nb_pixel_total : 37590 time to create 1 rle with old method : 0.0431668758392334 time for calcul the mask position with numpy : 0.024568557739257812 nb_pixel_total : 46240 time to create 1 rle with old method : 0.05331611633300781 time for calcul the mask position with numpy : 0.02400827407836914 nb_pixel_total : 91735 time to create 1 rle with old method : 0.10690736770629883 time for calcul the mask position with numpy : 0.026766538619995117 nb_pixel_total : 49419 time to create 1 rle with old method : 0.0777428150177002 time for calcul the mask position with numpy : 0.02553844451904297 nb_pixel_total : 57281 time to create 1 rle with old method : 0.08748626708984375 time for calcul the mask position with numpy : 0.023809194564819336 nb_pixel_total : 19954 time to create 1 rle with old method : 0.022787094116210938 create new chi : 1.872765302658081 time to delete rle : 0.0008616447448730469 batch 1 Loaded 15 chid ids of type : 3594 ++++++++Number RLEs to save : 5682 TO DO : save crop sub photo not yet done ! save time : 0.45542240142822266 nb_obj : 8 nb_hashtags : 3 time to prepare the origin masks : 4.498415470123291 time for calcul the mask position with numpy : 0.5584938526153564 nb_pixel_total : 7663533 time to create 1 rle with new method : 0.7937736511230469 time for calcul the mask position with numpy : 0.02378368377685547 nb_pixel_total : 74037 time to create 1 rle with old method : 0.08423733711242676 time for calcul the mask position with numpy : 0.023882627487182617 nb_pixel_total : 16266 time to create 1 rle with old method : 0.018645524978637695 time for calcul the mask position with numpy : 0.025246620178222656 nb_pixel_total : 177113 time to create 1 rle with new method : 0.6858541965484619 time for calcul the mask position with numpy : 0.024018526077270508 nb_pixel_total : 16530 time to create 1 rle with old method : 0.018839359283447266 time for calcul the mask position with numpy : 0.0247802734375 nb_pixel_total : 51073 time to create 1 rle with old method : 0.0590968132019043 time for calcul the mask position with numpy : 0.029465198516845703 nb_pixel_total : 167568 time to create 1 rle with new method : 0.6668210029602051 time for calcul the mask position with numpy : 0.02560257911682129 nb_pixel_total : 88362 time to create 1 rle with old method : 0.10139036178588867 time for calcul the mask position with numpy : 0.02588796615600586 nb_pixel_total : 39918 time to create 1 rle with old method : 0.04627490043640137 create new chi : 3.3335018157958984 time to delete rle : 0.001129150390625 batch 1 Loaded 17 chid ids of type : 3594 ++++++++++++Number RLEs to save : 7714 TO DO : save crop sub photo not yet done ! save time : 0.6197502613067627 nb_obj : 9 nb_hashtags : 3 time to prepare the origin masks : 4.028846263885498 time for calcul the mask position with numpy : 0.5738716125488281 nb_pixel_total : 7781078 time to create 1 rle with new method : 0.7880065441131592 time for calcul the mask position with numpy : 0.02396392822265625 nb_pixel_total : 42880 time to create 1 rle with old method : 0.04892086982727051 time for calcul the mask position with numpy : 0.024146318435668945 nb_pixel_total : 58891 time to create 1 rle with old method : 0.06808662414550781 time for calcul the mask position with numpy : 0.02605915069580078 nb_pixel_total : 67408 time to create 1 rle with old method : 0.0783393383026123 time for calcul the mask position with numpy : 0.02390909194946289 nb_pixel_total : 15324 time to create 1 rle with old method : 0.01752614974975586 time for calcul the mask position with numpy : 0.025400400161743164 nb_pixel_total : 34440 time to create 1 rle with old method : 0.03992414474487305 time for calcul the mask position with numpy : 0.026272296905517578 nb_pixel_total : 84902 time to create 1 rle with old method : 0.09607791900634766 time for calcul the mask position with numpy : 0.02487349510192871 nb_pixel_total : 28665 time to create 1 rle with old method : 0.032547950744628906 time for calcul the mask position with numpy : 0.025856494903564453 nb_pixel_total : 154427 time to create 1 rle with new method : 0.7621030807495117 time for calcul the mask position with numpy : 0.024284839630126953 nb_pixel_total : 26385 time to create 1 rle with old method : 0.029975414276123047 create new chi : 2.827101469039917 time to delete rle : 0.0010592937469482422 batch 1 Loaded 19 chid ids of type : 3594 +++++++++Number RLEs to save : 7424 TO DO : save crop sub photo not yet done ! save time : 0.5784611701965332 map_output_result : {1385303709: (0.0, 'Should be the crop_list due to order', 0), 1385303675: (0.0, 'Should be the crop_list due to order', 0), 1385303674: (0.0, 'Should be the crop_list due to order', 0), 1385303671: (0.0, 'Should be the crop_list due to order', 0), 1385303670: (0.0, 'Should be the crop_list due to order', 0), 1385303669: (0.0, 'Should be the crop_list due to order', 0), 1385303668: (0.0, 'Should be the crop_list due to order', 0), 1385303634: (0.0, 'Should be the crop_list due to order', 0), 1385303632: (0.0, 'Should be the crop_list due to order', 0), 1385303630: (0.0, 'Should be the crop_list due to order', 0), 1385303628: (0.0, 'Should be the crop_list due to order', 0), 1385303627: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1385303709, 1385303675, 1385303674, 1385303671, 1385303670, 1385303669, 1385303668, 1385303634, 1385303632, 1385303630, 1385303628, 1385303627] Looping around the photos to save general results len do output : 12 /1385303709.Didn't retrieve data . /1385303675.Didn't retrieve data . /1385303674.Didn't retrieve data . /1385303671.Didn't retrieve data . /1385303670.Didn't retrieve data . /1385303669.Didn't retrieve data . /1385303668.Didn't retrieve data . /1385303634.Didn't retrieve data . /1385303632.Didn't retrieve data . /1385303630.Didn't retrieve data . /1385303628.Didn't retrieve data . /1385303627.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303709', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303675', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303674', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303671', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303670', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303669', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303668', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303634', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303632', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303630', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303628', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303627', None, None, None, None, None, '3754488') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 36 time used for this insertion : 0.11636567115783691 save_final save missing photos in datou_result : time spend for datou_step_exec : 73.15233325958252 time spend to save output : 0.11790704727172852 total time spend for step 3 : 73.27024030685425 step4:ventilate_hashtags_in_portfolio Sun Sep 21 08:53:40 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 27071629 get user id for portfolio 27071629 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27071629 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('autre','pehd','metal','mal_croppe','pet_clair','background','carton','flou','papier','pet_fonce','environnement')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27071629 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('autre','pehd','metal','mal_croppe','pet_clair','background','carton','flou','papier','pet_fonce','environnement')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27071629 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('autre','pehd','metal','mal_croppe','pet_clair','background','carton','flou','papier','pet_fonce','environnement')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://marlene.fotonower.com/velours/27071708,27071709,27071710,27071711,27071712,27071713,27071714,27071715,27071716,27071717,27071718?tags=autre,pehd,metal,mal_croppe,pet_clair,background,carton,flou,papier,pet_fonce,environnement Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1385303709, 1385303675, 1385303674, 1385303671, 1385303670, 1385303669, 1385303668, 1385303634, 1385303632, 1385303630, 1385303628, 1385303627] Looping around the photos to save general results len do output : 1 /27071629. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303709', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303675', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303674', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303671', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303670', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303669', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303668', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303634', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303632', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303630', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303628', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303627', None, None, None, None, None, '3754488') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 13 time used for this insertion : 0.08315181732177734 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.8488733768463135 time spend to save output : 0.09871888160705566 total time spend for step 4 : 1.9475922584533691 step5:final Sun Sep 21 08:53:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1385303709: ('0.0415430290316358',), 1385303675: ('0.0415430290316358',), 1385303674: ('0.0415430290316358',), 1385303671: ('0.0415430290316358',), 1385303670: ('0.0415430290316358',), 1385303669: ('0.0415430290316358',), 1385303668: ('0.0415430290316358',), 1385303634: ('0.0415430290316358',), 1385303632: ('0.0415430290316358',), 1385303630: ('0.0415430290316358',), 1385303628: ('0.0415430290316358',), 1385303627: ('0.0415430290316358',)} new output for save of step final : {1385303709: ('0.0415430290316358',), 1385303675: ('0.0415430290316358',), 1385303674: ('0.0415430290316358',), 1385303671: ('0.0415430290316358',), 1385303670: ('0.0415430290316358',), 1385303669: ('0.0415430290316358',), 1385303668: ('0.0415430290316358',), 1385303634: ('0.0415430290316358',), 1385303632: ('0.0415430290316358',), 1385303630: ('0.0415430290316358',), 1385303628: ('0.0415430290316358',), 1385303627: ('0.0415430290316358',)} [1385303709, 1385303675, 1385303674, 1385303671, 1385303670, 1385303669, 1385303668, 1385303634, 1385303632, 1385303630, 1385303628, 1385303627] Looping around the photos to save general results len do output : 12 /1385303709.Didn't retrieve data . /1385303675.Didn't retrieve data . /1385303674.Didn't retrieve data . /1385303671.Didn't retrieve data . /1385303670.Didn't retrieve data . /1385303669.Didn't retrieve data . /1385303668.Didn't retrieve data . /1385303634.Didn't retrieve data . /1385303632.Didn't retrieve data . /1385303630.Didn't retrieve data . /1385303628.Didn't retrieve data . /1385303627.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303709', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303675', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303674', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303671', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303670', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303669', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303668', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303634', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303632', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303630', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303628', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303627', None, None, None, None, None, '3754488') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 36 time used for this insertion : 0.10034918785095215 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.15236520767211914 time spend to save output : 0.10100054740905762 total time spend for step 5 : 0.25336575508117676 step6:blur_detection Sun Sep 21 08:53:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1758437429_2145937_1385303709_6a2a2b31e6245e8e607dc03c1cd06948.jpg resize: (2160, 3840) 1385303709 -6.062703347493157 treat image : temp/1758437429_2145937_1385303675_a6a30a7ec9b9b6afd3908998425425e0.jpg resize: (2160, 3840) 1385303675 -6.996732223658671 treat image : temp/1758437429_2145937_1385303674_52334ed514c257fa0e5dfc49ecd1873f.jpg resize: (2160, 3840) 1385303674 -7.0827049026174596 treat image : temp/1758437429_2145937_1385303671_e80e8cdaea7fef38d7a6d9875df2a495.jpg resize: (2160, 3840) 1385303671 -7.089042032610837 treat image : temp/1758437429_2145937_1385303670_283eddf14a41f934758648aae4dbde57.jpg resize: (2160, 3840) 1385303670 -6.9583256929247845 treat image : temp/1758437429_2145937_1385303669_ef36019db65b1900dc7911ba6da376e7.jpg resize: (2160, 3840) 1385303669 -7.136817559440267 treat image : temp/1758437429_2145937_1385303668_1aabe0ecee95025370d68cdcb0278371.jpg resize: (2160, 3840) 1385303668 -7.119748836549801 treat image : temp/1758437429_2145937_1385303634_e46e9c3afa80133477edc151a014fe1c.jpg resize: (2160, 3840) 1385303634 -7.045864085361389 treat image : temp/1758437429_2145937_1385303632_fe9294c372ab6215d570ce4eb6cb21db.jpg resize: (2160, 3840) 1385303632 -6.993066938282997 treat image : temp/1758437429_2145937_1385303630_cb98f3ac7b8baca12f2885c6abf3d392.jpg resize: (2160, 3840) 1385303630 -6.955273557720944 treat image : temp/1758437429_2145937_1385303628_a5ad1d498e5e5bff826beb5012150e34.jpg resize: (2160, 3840) 1385303628 -7.022598496487383 treat image : temp/1758437429_2145937_1385303627_881fc7b4137673023c44f4bfee24d7b7.jpg resize: (2160, 3840) 1385303627 -6.996398149595581 treat image : temp/1758437429_2145937_1385303709_6a2a2b31e6245e8e607dc03c1cd06948_rle_crop_3969170197_0.png resize: (346, 149) 1385321200 -4.468674089784971 treat image : temp/1758437429_2145937_1385303709_6a2a2b31e6245e8e607dc03c1cd06948_rle_crop_3969170198_0.png resize: (128, 261) 1385321202 -4.77519665414613 treat image : temp/1758437429_2145937_1385303709_6a2a2b31e6245e8e607dc03c1cd06948_rle_crop_3969170200_0.png resize: (299, 403) 1385321203 -4.108802885932975 treat image : temp/1758437429_2145937_1385303675_a6a30a7ec9b9b6afd3908998425425e0_rle_crop_3969170206_0.png resize: (224, 235) 1385321204 -3.986630098850369 treat image : temp/1758437429_2145937_1385303675_a6a30a7ec9b9b6afd3908998425425e0_rle_crop_3969170207_0.png resize: (239, 298) 1385321205 -4.426950745233979 treat image : temp/1758437429_2145937_1385303675_a6a30a7ec9b9b6afd3908998425425e0_rle_crop_3969170212_0.png resize: (221, 279) 1385321206 -4.064049550448206 treat image : temp/1758437429_2145937_1385303674_52334ed514c257fa0e5dfc49ecd1873f_rle_crop_3969170221_0.png resize: (127, 89) 1385321207 -3.398055612514911 treat image : temp/1758437429_2145937_1385303671_e80e8cdaea7fef38d7a6d9875df2a495_rle_crop_3969170224_0.png resize: (397, 153) 1385321208 -4.2272279199205585 treat image : temp/1758437429_2145937_1385303671_e80e8cdaea7fef38d7a6d9875df2a495_rle_crop_3969170227_0.png resize: (167, 195) 1385321209 -3.669098308587193 treat image : temp/1758437429_2145937_1385303669_ef36019db65b1900dc7911ba6da376e7_rle_crop_3969170233_0.png resize: (206, 209) 1385321210 -4.965131289855655 treat image : temp/1758437429_2145937_1385303668_1aabe0ecee95025370d68cdcb0278371_rle_crop_3969170235_0.png resize: (132, 132) 1385321211 -3.9262968784767436 treat image : temp/1758437429_2145937_1385303668_1aabe0ecee95025370d68cdcb0278371_rle_crop_3969170236_0.png resize: (326, 441) 1385321212 -5.446284698395807 treat image : temp/1758437429_2145937_1385303634_e46e9c3afa80133477edc151a014fe1c_rle_crop_3969170241_0.png resize: (248, 271) 1385321213 -4.184491938738908 treat image : temp/1758437429_2145937_1385303632_fe9294c372ab6215d570ce4eb6cb21db_rle_crop_3969170245_0.png resize: (205, 204) 1385321214 -4.019076934722254 treat image : temp/1758437429_2145937_1385303632_fe9294c372ab6215d570ce4eb6cb21db_rle_crop_3969170250_0.png resize: (75, 252) 1385321215 -4.927573951976114 treat image : temp/1758437429_2145937_1385303630_cb98f3ac7b8baca12f2885c6abf3d392_rle_crop_3969170252_0.png resize: (239, 415) 1385321216 -3.320994021781182 treat image : temp/1758437429_2145937_1385303630_cb98f3ac7b8baca12f2885c6abf3d392_rle_crop_3969170253_0.png resize: (212, 284) 1385321217 -4.7942738453182265 treat image : temp/1758437429_2145937_1385303630_cb98f3ac7b8baca12f2885c6abf3d392_rle_crop_3969170255_0.png resize: (180, 348) 1385321218 -4.406393549050336 treat image : temp/1758437429_2145937_1385303630_cb98f3ac7b8baca12f2885c6abf3d392_rle_crop_3969170256_0.png resize: (432, 189) 1385321219 -4.540602525030511 treat image : temp/1758437429_2145937_1385303630_cb98f3ac7b8baca12f2885c6abf3d392_rle_crop_3969170257_0.png resize: (111, 227) 1385321220 -4.513144249312005 treat image : temp/1758437429_2145937_1385303628_a5ad1d498e5e5bff826beb5012150e34_rle_crop_3969170258_0.png resize: (239, 246) 1385321221 -3.785385829121366 treat image : temp/1758437429_2145937_1385303628_a5ad1d498e5e5bff826beb5012150e34_rle_crop_3969170259_0.png resize: (266, 440) 1385321222 -5.293120979234062 treat image : temp/1758437429_2145937_1385303628_a5ad1d498e5e5bff826beb5012150e34_rle_crop_3969170261_0.png resize: (232, 300) 1385321223 -3.205161823172379 treat image : temp/1758437429_2145937_1385303628_a5ad1d498e5e5bff826beb5012150e34_rle_crop_3969170263_0.png resize: (601, 465) 1385321224 -4.912283912579865 treat image : temp/1758437429_2145937_1385303628_a5ad1d498e5e5bff826beb5012150e34_rle_crop_3969170264_0.png resize: (341, 125) 1385321225 -4.098527650582642 treat image : temp/1758437429_2145937_1385303627_881fc7b4137673023c44f4bfee24d7b7_rle_crop_3969170266_0.png resize: (223, 195) 1385321226 -3.8243304343536746 treat image : temp/1758437429_2145937_1385303627_881fc7b4137673023c44f4bfee24d7b7_rle_crop_3969170267_0.png resize: (346, 701) 1385321227 -4.070366104799797 treat image : temp/1758437429_2145937_1385303627_881fc7b4137673023c44f4bfee24d7b7_rle_crop_3969170270_0.png resize: (309, 230) 1385321228 -3.140695643539598 treat image : temp/1758437429_2145937_1385303627_881fc7b4137673023c44f4bfee24d7b7_rle_crop_3969170272_0.png resize: (436, 251) 1385321229 -4.097958970001777 treat image : temp/1758437429_2145937_1385303627_881fc7b4137673023c44f4bfee24d7b7_rle_crop_3969170274_0.png resize: (281, 215) 1385321230 -3.98809514080995 treat image : temp/1758437429_2145937_1385303709_6a2a2b31e6245e8e607dc03c1cd06948_rle_crop_3969170201_0.png resize: (217, 234) 1385321234 -3.554847449753294 treat image : temp/1758437429_2145937_1385303709_6a2a2b31e6245e8e607dc03c1cd06948_rle_crop_3969170202_0.png resize: (611, 854) 1385321235 -5.311094187221531 treat image : temp/1758437429_2145937_1385303675_a6a30a7ec9b9b6afd3908998425425e0_rle_crop_3969170205_0.png resize: (179, 286) 1385321236 -3.0738759408870555 treat image : temp/1758437429_2145937_1385303675_a6a30a7ec9b9b6afd3908998425425e0_rle_crop_3969170209_0.png resize: (173, 143) 1385321237 -4.174456091738703 treat image : temp/1758437429_2145937_1385303675_a6a30a7ec9b9b6afd3908998425425e0_rle_crop_3969170214_0.png resize: (190, 146) 1385321238 -3.2612508021156676 treat image : temp/1758437429_2145937_1385303674_52334ed514c257fa0e5dfc49ecd1873f_rle_crop_3969170215_0.png resize: (235, 357) 1385321239 -4.726834471763739 treat image : temp/1758437429_2145937_1385303674_52334ed514c257fa0e5dfc49ecd1873f_rle_crop_3969170218_0.png resize: (171, 196) 1385321240 -3.8474320179019883 treat image : temp/1758437429_2145937_1385303674_52334ed514c257fa0e5dfc49ecd1873f_rle_crop_3969170219_0.png resize: (228, 170) 1385321241 -2.907890650144286 treat image : temp/1758437429_2145937_1385303671_e80e8cdaea7fef38d7a6d9875df2a495_rle_crop_3969170226_0.png resize: (355, 330) 1385321242 -3.3258700063216193 treat image : temp/1758437429_2145937_1385303668_1aabe0ecee95025370d68cdcb0278371_rle_crop_3969170237_0.png resize: (373, 494) 1385321243 -5.482079164892299 treat image : temp/1758437429_2145937_1385303634_e46e9c3afa80133477edc151a014fe1c_rle_crop_3969170239_0.png resize: (86, 126) 1385321244 -1.8936025894232056 treat image : temp/1758437429_2145937_1385303634_e46e9c3afa80133477edc151a014fe1c_rle_crop_3969170240_0.png resize: (163, 250) 1385321245 -3.173506371813693 treat image : temp/1758437429_2145937_1385303632_fe9294c372ab6215d570ce4eb6cb21db_rle_crop_3969170246_0.png resize: (356, 221) 1385321246 -1.957547460300565 treat image : temp/1758437429_2145937_1385303675_a6a30a7ec9b9b6afd3908998425425e0_rle_crop_3969170208_0.png resize: (135, 96) 1385321248 -4.417641216139478 treat image : temp/1758437429_2145937_1385303674_52334ed514c257fa0e5dfc49ecd1873f_rle_crop_3969170220_0.png resize: (254, 217) 1385321249 -3.935217943228546 treat image : temp/1758437429_2145937_1385303670_283eddf14a41f934758648aae4dbde57_rle_crop_3969170229_0.png resize: (131, 132) 1385321250 -4.248376981730427 treat image : temp/1758437429_2145937_1385303628_a5ad1d498e5e5bff826beb5012150e34_rle_crop_3969170262_0.png resize: (131, 155) 1385321251 -4.682734719876641 treat image : temp/1758437429_2145937_1385303709_6a2a2b31e6245e8e607dc03c1cd06948_rle_crop_3969170203_0.png resize: (343, 364) 1385321268 -4.657080790684045 treat image : temp/1758437429_2145937_1385303675_a6a30a7ec9b9b6afd3908998425425e0_rle_crop_3969170204_0.png resize: (257, 321) 1385321269 -5.5108603416228465 treat image : temp/1758437429_2145937_1385303675_a6a30a7ec9b9b6afd3908998425425e0_rle_crop_3969170211_0.png resize: (394, 318) 1385321270 -4.789330220454067 treat image : temp/1758437429_2145937_1385303675_a6a30a7ec9b9b6afd3908998425425e0_rle_crop_3969170213_0.png resize: (288, 361) 1385321271 -3.651402957438182 treat image : temp/1758437429_2145937_1385303674_52334ed514c257fa0e5dfc49ecd1873f_rle_crop_3969170216_0.png resize: (146, 107) 1385321272 -3.784676693073289 treat image : temp/1758437429_2145937_1385303674_52334ed514c257fa0e5dfc49ecd1873f_rle_crop_3969170217_0.png resize: (332, 433) 1385321273 -5.693867919656775 treat image : temp/1758437429_2145937_1385303671_e80e8cdaea7fef38d7a6d9875df2a495_rle_crop_3969170222_0.png resize: (234, 423) 1385321274 -4.670769251839949 treat image : temp/1758437429_2145937_1385303671_e80e8cdaea7fef38d7a6d9875df2a495_rle_crop_3969170223_0.png resize: (216, 264) 1385321275 -4.788014307538683 treat image : temp/1758437429_2145937_1385303671_e80e8cdaea7fef38d7a6d9875df2a495_rle_crop_3969170225_0.png resize: (113, 160) 1385321276 -4.533770984497853 treat image : temp/1758437429_2145937_1385303670_283eddf14a41f934758648aae4dbde57_rle_crop_3969170230_0.png resize: (252, 343) 1385321277 -5.112768251712717 treat image : temp/1758437429_2145937_1385303669_ef36019db65b1900dc7911ba6da376e7_rle_crop_3969170231_0.png resize: (244, 420) 1385321278 -6.257108480692933 treat image : temp/1758437429_2145937_1385303669_ef36019db65b1900dc7911ba6da376e7_rle_crop_3969170232_0.png resize: (418, 242) 1385321279 -5.381690651710873 treat image : temp/1758437429_2145937_1385303669_ef36019db65b1900dc7911ba6da376e7_rle_crop_3969170234_0.png resize: (125, 194) 1385321280 -3.2716639876147493 treat image : temp/1758437429_2145937_1385303634_e46e9c3afa80133477edc151a014fe1c_rle_crop_3969170238_0.png resize: (231, 261) 1385321281 -4.456246236521479 treat image : temp/1758437429_2145937_1385303634_e46e9c3afa80133477edc151a014fe1c_rle_crop_3969170242_0.png resize: (256, 278) 1385321282 -4.8749423302284205 treat image : temp/1758437429_2145937_1385303632_fe9294c372ab6215d570ce4eb6cb21db_rle_crop_3969170243_0.png resize: (248, 205) 1385321283 -4.347247205426504 treat image : temp/1758437429_2145937_1385303632_fe9294c372ab6215d570ce4eb6cb21db_rle_crop_3969170244_0.png resize: (323, 262) 1385321284 -3.9351806179955386 treat image : temp/1758437429_2145937_1385303632_fe9294c372ab6215d570ce4eb6cb21db_rle_crop_3969170247_0.png resize: (284, 360) 1385321285 -4.523408469378315 treat image : temp/1758437429_2145937_1385303632_fe9294c372ab6215d570ce4eb6cb21db_rle_crop_3969170248_0.png resize: (234, 212) 1385321286 -4.558460029774132 treat image : temp/1758437429_2145937_1385303632_fe9294c372ab6215d570ce4eb6cb21db_rle_crop_3969170249_0.png resize: (324, 360) 1385321287 -4.751246667117105 treat image : temp/1758437429_2145937_1385303630_cb98f3ac7b8baca12f2885c6abf3d392_rle_crop_3969170254_0.png resize: (252, 440) 1385321288 -5.0000052527195065 treat image : temp/1758437429_2145937_1385303628_a5ad1d498e5e5bff826beb5012150e34_rle_crop_3969170260_0.png resize: (394, 531) 1385321289 -4.973281358228175 treat image : temp/1758437429_2145937_1385303628_a5ad1d498e5e5bff826beb5012150e34_rle_crop_3969170265_0.png resize: (280, 344) 1385321290 -5.213335814244356 treat image : temp/1758437429_2145937_1385303627_881fc7b4137673023c44f4bfee24d7b7_rle_crop_3969170269_0.png resize: (297, 395) 1385321291 -4.74767035600189 treat image : temp/1758437429_2145937_1385303627_881fc7b4137673023c44f4bfee24d7b7_rle_crop_3969170273_0.png resize: (306, 259) 1385321292 -5.8164798469749694 treat image : temp/1758437429_2145937_1385303709_6a2a2b31e6245e8e607dc03c1cd06948_rle_crop_3969170199_0.png resize: (114, 174) 1385321295 -1.8610174325685473 treat image : temp/1758437429_2145937_1385303670_283eddf14a41f934758648aae4dbde57_rle_crop_3969170228_0.png resize: (318, 321) 1385321296 -5.088831771122038 treat image : temp/1758437429_2145937_1385303630_cb98f3ac7b8baca12f2885c6abf3d392_rle_crop_3969170251_0.png resize: (144, 162) 1385321297 -3.978391277482126 treat image : temp/1758437429_2145937_1385303627_881fc7b4137673023c44f4bfee24d7b7_rle_crop_3969170268_0.png resize: (212, 193) 1385321298 -4.842205275602362 treat image : temp/1758437429_2145937_1385303627_881fc7b4137673023c44f4bfee24d7b7_rle_crop_3969170271_0.png resize: (120, 148) 1385321299 -4.757023044617261 treat image : temp/1758437429_2145937_1385303675_a6a30a7ec9b9b6afd3908998425425e0_rle_crop_3969170210_0.png resize: (269, 410) 1385321301 -6.256893754332424 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 90 time used for this insertion : 0.016187191009521484 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 90 time used for this insertion : 0.33147478103637695 save missing photos in datou_result : time spend for datou_step_exec : 44.27988791465759 time spend to save output : 0.3536555767059326 total time spend for step 6 : 44.633543491363525 step7:brightness Sun Sep 21 08:54:26 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1758437429_2145937_1385303709_6a2a2b31e6245e8e607dc03c1cd06948.jpg treat image : temp/1758437429_2145937_1385303675_a6a30a7ec9b9b6afd3908998425425e0.jpg treat image : temp/1758437429_2145937_1385303674_52334ed514c257fa0e5dfc49ecd1873f.jpg treat image : temp/1758437429_2145937_1385303671_e80e8cdaea7fef38d7a6d9875df2a495.jpg treat image : temp/1758437429_2145937_1385303670_283eddf14a41f934758648aae4dbde57.jpg treat image : temp/1758437429_2145937_1385303669_ef36019db65b1900dc7911ba6da376e7.jpg treat image : temp/1758437429_2145937_1385303668_1aabe0ecee95025370d68cdcb0278371.jpg treat image : temp/1758437429_2145937_1385303634_e46e9c3afa80133477edc151a014fe1c.jpg treat image : temp/1758437429_2145937_1385303632_fe9294c372ab6215d570ce4eb6cb21db.jpg treat image : temp/1758437429_2145937_1385303630_cb98f3ac7b8baca12f2885c6abf3d392.jpg treat image : temp/1758437429_2145937_1385303628_a5ad1d498e5e5bff826beb5012150e34.jpg treat image : temp/1758437429_2145937_1385303627_881fc7b4137673023c44f4bfee24d7b7.jpg treat image : temp/1758437429_2145937_1385303709_6a2a2b31e6245e8e607dc03c1cd06948_rle_crop_3969170197_0.png treat image : temp/1758437429_2145937_1385303709_6a2a2b31e6245e8e607dc03c1cd06948_rle_crop_3969170198_0.png treat image : temp/1758437429_2145937_1385303709_6a2a2b31e6245e8e607dc03c1cd06948_rle_crop_3969170200_0.png treat image : temp/1758437429_2145937_1385303675_a6a30a7ec9b9b6afd3908998425425e0_rle_crop_3969170206_0.png treat image : temp/1758437429_2145937_1385303675_a6a30a7ec9b9b6afd3908998425425e0_rle_crop_3969170207_0.png treat image : temp/1758437429_2145937_1385303675_a6a30a7ec9b9b6afd3908998425425e0_rle_crop_3969170212_0.png treat image : temp/1758437429_2145937_1385303674_52334ed514c257fa0e5dfc49ecd1873f_rle_crop_3969170221_0.png treat image : temp/1758437429_2145937_1385303671_e80e8cdaea7fef38d7a6d9875df2a495_rle_crop_3969170224_0.png treat image : temp/1758437429_2145937_1385303671_e80e8cdaea7fef38d7a6d9875df2a495_rle_crop_3969170227_0.png treat image : temp/1758437429_2145937_1385303669_ef36019db65b1900dc7911ba6da376e7_rle_crop_3969170233_0.png treat image : temp/1758437429_2145937_1385303668_1aabe0ecee95025370d68cdcb0278371_rle_crop_3969170235_0.png treat image : temp/1758437429_2145937_1385303668_1aabe0ecee95025370d68cdcb0278371_rle_crop_3969170236_0.png treat image : temp/1758437429_2145937_1385303634_e46e9c3afa80133477edc151a014fe1c_rle_crop_3969170241_0.png treat image : temp/1758437429_2145937_1385303632_fe9294c372ab6215d570ce4eb6cb21db_rle_crop_3969170245_0.png treat image : temp/1758437429_2145937_1385303632_fe9294c372ab6215d570ce4eb6cb21db_rle_crop_3969170250_0.png treat image : temp/1758437429_2145937_1385303630_cb98f3ac7b8baca12f2885c6abf3d392_rle_crop_3969170252_0.png treat image : temp/1758437429_2145937_1385303630_cb98f3ac7b8baca12f2885c6abf3d392_rle_crop_3969170253_0.png treat image : temp/1758437429_2145937_1385303630_cb98f3ac7b8baca12f2885c6abf3d392_rle_crop_3969170255_0.png treat image : temp/1758437429_2145937_1385303630_cb98f3ac7b8baca12f2885c6abf3d392_rle_crop_3969170256_0.png treat image : temp/1758437429_2145937_1385303630_cb98f3ac7b8baca12f2885c6abf3d392_rle_crop_3969170257_0.png treat image : temp/1758437429_2145937_1385303628_a5ad1d498e5e5bff826beb5012150e34_rle_crop_3969170258_0.png treat image : temp/1758437429_2145937_1385303628_a5ad1d498e5e5bff826beb5012150e34_rle_crop_3969170259_0.png treat image : temp/1758437429_2145937_1385303628_a5ad1d498e5e5bff826beb5012150e34_rle_crop_3969170261_0.png treat image : temp/1758437429_2145937_1385303628_a5ad1d498e5e5bff826beb5012150e34_rle_crop_3969170263_0.png treat image : temp/1758437429_2145937_1385303628_a5ad1d498e5e5bff826beb5012150e34_rle_crop_3969170264_0.png treat image : temp/1758437429_2145937_1385303627_881fc7b4137673023c44f4bfee24d7b7_rle_crop_3969170266_0.png treat image : temp/1758437429_2145937_1385303627_881fc7b4137673023c44f4bfee24d7b7_rle_crop_3969170267_0.png treat image : temp/1758437429_2145937_1385303627_881fc7b4137673023c44f4bfee24d7b7_rle_crop_3969170270_0.png treat image : temp/1758437429_2145937_1385303627_881fc7b4137673023c44f4bfee24d7b7_rle_crop_3969170272_0.png treat image : temp/1758437429_2145937_1385303627_881fc7b4137673023c44f4bfee24d7b7_rle_crop_3969170274_0.png treat image : temp/1758437429_2145937_1385303709_6a2a2b31e6245e8e607dc03c1cd06948_rle_crop_3969170201_0.png treat image : temp/1758437429_2145937_1385303709_6a2a2b31e6245e8e607dc03c1cd06948_rle_crop_3969170202_0.png treat image : temp/1758437429_2145937_1385303675_a6a30a7ec9b9b6afd3908998425425e0_rle_crop_3969170205_0.png treat image : temp/1758437429_2145937_1385303675_a6a30a7ec9b9b6afd3908998425425e0_rle_crop_3969170209_0.png treat image : temp/1758437429_2145937_1385303675_a6a30a7ec9b9b6afd3908998425425e0_rle_crop_3969170214_0.png treat image : temp/1758437429_2145937_1385303674_52334ed514c257fa0e5dfc49ecd1873f_rle_crop_3969170215_0.png treat image : temp/1758437429_2145937_1385303674_52334ed514c257fa0e5dfc49ecd1873f_rle_crop_3969170218_0.png treat image : temp/1758437429_2145937_1385303674_52334ed514c257fa0e5dfc49ecd1873f_rle_crop_3969170219_0.png treat image : temp/1758437429_2145937_1385303671_e80e8cdaea7fef38d7a6d9875df2a495_rle_crop_3969170226_0.png treat image : temp/1758437429_2145937_1385303668_1aabe0ecee95025370d68cdcb0278371_rle_crop_3969170237_0.png treat image : temp/1758437429_2145937_1385303634_e46e9c3afa80133477edc151a014fe1c_rle_crop_3969170239_0.png treat image : temp/1758437429_2145937_1385303634_e46e9c3afa80133477edc151a014fe1c_rle_crop_3969170240_0.png treat image : temp/1758437429_2145937_1385303632_fe9294c372ab6215d570ce4eb6cb21db_rle_crop_3969170246_0.png treat image : temp/1758437429_2145937_1385303675_a6a30a7ec9b9b6afd3908998425425e0_rle_crop_3969170208_0.png treat image : temp/1758437429_2145937_1385303674_52334ed514c257fa0e5dfc49ecd1873f_rle_crop_3969170220_0.png treat image : temp/1758437429_2145937_1385303670_283eddf14a41f934758648aae4dbde57_rle_crop_3969170229_0.png treat image : temp/1758437429_2145937_1385303628_a5ad1d498e5e5bff826beb5012150e34_rle_crop_3969170262_0.png treat image : temp/1758437429_2145937_1385303709_6a2a2b31e6245e8e607dc03c1cd06948_rle_crop_3969170203_0.png treat image : temp/1758437429_2145937_1385303675_a6a30a7ec9b9b6afd3908998425425e0_rle_crop_3969170204_0.png treat image : temp/1758437429_2145937_1385303675_a6a30a7ec9b9b6afd3908998425425e0_rle_crop_3969170211_0.png treat image : temp/1758437429_2145937_1385303675_a6a30a7ec9b9b6afd3908998425425e0_rle_crop_3969170213_0.png treat image : temp/1758437429_2145937_1385303674_52334ed514c257fa0e5dfc49ecd1873f_rle_crop_3969170216_0.png treat image : temp/1758437429_2145937_1385303674_52334ed514c257fa0e5dfc49ecd1873f_rle_crop_3969170217_0.png treat image : temp/1758437429_2145937_1385303671_e80e8cdaea7fef38d7a6d9875df2a495_rle_crop_3969170222_0.png treat image : temp/1758437429_2145937_1385303671_e80e8cdaea7fef38d7a6d9875df2a495_rle_crop_3969170223_0.png treat image : temp/1758437429_2145937_1385303671_e80e8cdaea7fef38d7a6d9875df2a495_rle_crop_3969170225_0.png treat image : temp/1758437429_2145937_1385303670_283eddf14a41f934758648aae4dbde57_rle_crop_3969170230_0.png treat image : temp/1758437429_2145937_1385303669_ef36019db65b1900dc7911ba6da376e7_rle_crop_3969170231_0.png treat image : temp/1758437429_2145937_1385303669_ef36019db65b1900dc7911ba6da376e7_rle_crop_3969170232_0.png treat image : temp/1758437429_2145937_1385303669_ef36019db65b1900dc7911ba6da376e7_rle_crop_3969170234_0.png treat image : temp/1758437429_2145937_1385303634_e46e9c3afa80133477edc151a014fe1c_rle_crop_3969170238_0.png treat image : temp/1758437429_2145937_1385303634_e46e9c3afa80133477edc151a014fe1c_rle_crop_3969170242_0.png treat image : temp/1758437429_2145937_1385303632_fe9294c372ab6215d570ce4eb6cb21db_rle_crop_3969170243_0.png treat image : temp/1758437429_2145937_1385303632_fe9294c372ab6215d570ce4eb6cb21db_rle_crop_3969170244_0.png treat image : temp/1758437429_2145937_1385303632_fe9294c372ab6215d570ce4eb6cb21db_rle_crop_3969170247_0.png treat image : temp/1758437429_2145937_1385303632_fe9294c372ab6215d570ce4eb6cb21db_rle_crop_3969170248_0.png treat image : temp/1758437429_2145937_1385303632_fe9294c372ab6215d570ce4eb6cb21db_rle_crop_3969170249_0.png treat image : temp/1758437429_2145937_1385303630_cb98f3ac7b8baca12f2885c6abf3d392_rle_crop_3969170254_0.png treat image : temp/1758437429_2145937_1385303628_a5ad1d498e5e5bff826beb5012150e34_rle_crop_3969170260_0.png treat image : temp/1758437429_2145937_1385303628_a5ad1d498e5e5bff826beb5012150e34_rle_crop_3969170265_0.png treat image : temp/1758437429_2145937_1385303627_881fc7b4137673023c44f4bfee24d7b7_rle_crop_3969170269_0.png treat image : temp/1758437429_2145937_1385303627_881fc7b4137673023c44f4bfee24d7b7_rle_crop_3969170273_0.png treat image : temp/1758437429_2145937_1385303709_6a2a2b31e6245e8e607dc03c1cd06948_rle_crop_3969170199_0.png treat image : temp/1758437429_2145937_1385303670_283eddf14a41f934758648aae4dbde57_rle_crop_3969170228_0.png treat image : temp/1758437429_2145937_1385303630_cb98f3ac7b8baca12f2885c6abf3d392_rle_crop_3969170251_0.png treat image : temp/1758437429_2145937_1385303627_881fc7b4137673023c44f4bfee24d7b7_rle_crop_3969170268_0.png treat image : temp/1758437429_2145937_1385303627_881fc7b4137673023c44f4bfee24d7b7_rle_crop_3969170271_0.png treat image : temp/1758437429_2145937_1385303675_a6a30a7ec9b9b6afd3908998425425e0_rle_crop_3969170210_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 90 time used for this insertion : 0.019643068313598633 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 90 time used for this insertion : 0.05712723731994629 save missing photos in datou_result : time spend for datou_step_exec : 11.64832067489624 time spend to save output : 0.08344864845275879 total time spend for step 7 : 11.731769323348999 step8:velours_tree Sun Sep 21 08:54:38 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.28629446029663086 time spend to save output : 5.054473876953125e-05 total time spend for step 8 : 0.2863450050354004 step9:send_mail_cod Sun Sep 21 08:54:38 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P27071629_21-09-2025_08_54_38.pdf 27071708 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette270717081758437678 27071709 imagette270717091758437679 27071710 change filename to text .change filename to text .change filename to text .change filename to text .imagette270717101758437679 27071711 imagette270717111758437679 27071712 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette270717121758437679 27071713 imagette270717131758437681 27071714 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette270717141758437681 27071715 imagette270717151758437682 27071716 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette270717161758437682 27071717 change filename to text .imagette270717171758437684 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=27071629 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://marlene.fotonower.com/velours/27071708,27071709,27071710,27071711,27071712,27071713,27071714,27071715,27071716,27071717,27071718?tags=autre,pehd,metal,mal_croppe,pet_clair,background,carton,flou,papier,pet_fonce,environnement args[1385303709] : ((1385303709, -6.062703347493157, 492609224), (1385303709, -0.3791155735131028, 496442774), '0.0415430290316358') We are sending mail with results at report@fotonower.com args[1385303675] : ((1385303675, -6.996732223658671, 492609224), (1385303675, 0.015360356165488812, 2107752395), '0.0415430290316358') We are sending mail with results at report@fotonower.com args[1385303674] : ((1385303674, -7.0827049026174596, 492609224), (1385303674, 0.002186312412055928, 2107752395), '0.0415430290316358') We are sending mail with results at report@fotonower.com args[1385303671] : ((1385303671, -7.089042032610837, 492609224), (1385303671, 0.04931349624009493, 2107752395), '0.0415430290316358') We are sending mail with results at report@fotonower.com args[1385303670] : ((1385303670, -6.9583256929247845, 492609224), (1385303670, -0.00174995876241235, 2107752395), '0.0415430290316358') We are sending mail with results at report@fotonower.com args[1385303669] : ((1385303669, -7.136817559440267, 492609224), (1385303669, 0.06343339923198493, 2107752395), '0.0415430290316358') We are sending mail with results at report@fotonower.com args[1385303668] : ((1385303668, -7.119748836549801, 492609224), (1385303668, 0.209284682964363, 2107752395), '0.0415430290316358') We are sending mail with results at report@fotonower.com args[1385303634] : ((1385303634, -7.045864085361389, 492609224), (1385303634, -0.08092924136471445, 496442774), '0.0415430290316358') We are sending mail with results at report@fotonower.com args[1385303632] : ((1385303632, -6.993066938282997, 492609224), (1385303632, 0.05293727737229081, 2107752395), '0.0415430290316358') We are sending mail with results at report@fotonower.com args[1385303630] : ((1385303630, -6.955273557720944, 492609224), (1385303630, -0.01153518016163978, 2107752395), '0.0415430290316358') We are sending mail with results at report@fotonower.com args[1385303628] : ((1385303628, -7.022598496487383, 492609224), (1385303628, 0.11899967438208543, 2107752395), '0.0415430290316358') We are sending mail with results at report@fotonower.com args[1385303627] : ((1385303627, -6.996398149595581, 492609224), (1385303627, 0.026372835296884535, 2107752395), '0.0415430290316358') We are sending mail with results at report@fotonower.com refus_total : 0.0415430290316358 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=27071629 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27071629_21-09-2025_08_54_38.pdf results_Auto_P27071629_21-09-2025_08_54_38.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27071629_21-09-2025_08_54_38.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','27071629','results_Auto_P27071629_21-09-2025_08_54_38.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27071629_21-09-2025_08_54_38.pdf','pdf','','0.88','0.0415430290316358') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/27071629

https://www.fotonower.com/image?json=false&list_photos_id=1385303709
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1385303675
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1385303674
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1385303671
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1385303670
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1385303669
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1385303668
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1385303634
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1385303632
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1385303630
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1385303628
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1385303627
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 4.15%
Veuillez trouver les photos des contaminants.

exemples de contaminants: autre: https://www.fotonower.com/view/27071708?limit=200
exemples de contaminants: metal: https://www.fotonower.com/view/27071710?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/27071712?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/27071714?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/27071716?limit=200
exemples de contaminants: pet_fonce: https://www.fotonower.com/view/27071717?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27071629_21-09-2025_08_54_38.pdf.

Lien vers velours :https://marlene.fotonower.com/velours/27071708,27071709,27071710,27071711,27071712,27071713,27071714,27071715,27071716,27071717,27071718?tags=autre,pehd,metal,mal_croppe,pet_clair,background,carton,flou,papier,pet_fonce,environnement.


L'équipe Fotonower 202 b'' Server: nginx Date: Sun, 21 Sep 2025 06:54:48 GMT Content-Length: 0 Connection: close X-Message-Id: mXwHxRCPRQOaBwX-f_Tcww Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1385303709, 1385303675, 1385303674, 1385303671, 1385303670, 1385303669, 1385303668, 1385303634, 1385303632, 1385303630, 1385303628, 1385303627] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303709', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303675', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303674', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303671', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303670', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303669', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303668', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303634', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303632', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303630', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303628', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303627', None, None, None, None, None, '3754488') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 12 time used for this insertion : 0.09816622734069824 save_final save missing photos in datou_result : time spend for datou_step_exec : 9.501543998718262 time spend to save output : 0.0984184741973877 total time spend for step 9 : 9.59996247291565 step10:split_time_score Sun Sep 21 08:54:48 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('19', 12),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 18092025 27071629 Nombre de photos uploadées : 12 / 23040 (0%) 18092025 27071629 Nombre de photos taguées (types de déchets): 0 / 12 (0%) 18092025 27071629 Nombre de photos taguées (volume) : 0 / 12 (0%) elapsed_time : load_data_split_time_score 1.430511474609375e-06 elapsed_time : order_list_meta_photo_and_scores 5.245208740234375e-06 ???????????? elapsed_time : fill_and_build_computed_from_old_data 0.000989675521850586 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.5296483039855957 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.018575029205708013 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27016170_19-09-2025_00_17_07.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27016170 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27016170 AND mptpi.`type`=3726 To do Qualite : 0.013265808889117832 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27024177_19-09-2025_05_16_51.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27024177 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27024177 AND mptpi.`type`=3726 To do Qualite : 0.03832716212614271 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27066975_20-09-2025_08_57_10.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27066975 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27066975 AND mptpi.`type`=3726 To do Qualite : 0.013107935211456968 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27067010_20-09-2025_10_20_29.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27067010 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27067010 AND mptpi.`type`=3726 To do Qualite : 0.0398856488156835 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27067011_20-09-2025_08_50_06.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27067011 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27067011 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27071589 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27071590 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27071596 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27071598 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27071602 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27071604 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27071608 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27071618 order by id desc limit 1 Qualite : 0.02148846688075446 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27071625_21-09-2025_08_53_46.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27071625 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27071625 AND mptpi.`type`=3726 To do Qualite : 0.0415430290316358 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27071629_21-09-2025_08_54_38.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27071629 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27071629 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'18092025': {'nb_upload': 12, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1385303709, 1385303675, 1385303674, 1385303671, 1385303670, 1385303669, 1385303668, 1385303634, 1385303632, 1385303630, 1385303628, 1385303627] Looping around the photos to save general results len do output : 1 /27071629Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303709', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303675', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303674', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303671', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303670', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303669', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303668', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303634', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303632', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303630', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303628', None, None, None, None, None, '3754488') ('3318', None, None, None, None, None, None, None, '3754488') ('3318', '27071629', '1385303627', None, None, None, None, None, '3754488') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 13 time used for this insertion : 0.01850128173828125 save_final save missing photos in datou_result : time spend for datou_step_exec : 9.514087915420532 time spend to save output : 0.018761396408081055 total time spend for step 10 : 9.532849311828613 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 12 set_done_treatment 121.62user 101.51system 4:32.69elapsed 81%CPU (0avgtext+0avgdata 4409288maxresident)k 940968inputs+119816outputs (18785major+10958518minor)pagefaults 0swaps