python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 1282424 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['3568988'] with mtr_portfolio_ids : ['26121460'] and first list_photo_ids : [] new path : /proc/1282424/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 16 ; length of list_pids : 16 ; length of list_args : 16 time to download the photos : 2.0844268798828125 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Thu Aug 21 22:10:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10615 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-08-21 22:10:32.032325: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-08-21 22:10:32.041506: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-08-21 22:10:32.043008: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fa2b8000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-08-21 22:10:32.043058: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-08-21 22:10:32.045680: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-08-21 22:10:32.206977: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x214c68b0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-08-21 22:10:32.207041: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-08-21 22:10:32.208485: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-21 22:10:32.208935: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-21 22:10:32.212179: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-21 22:10:32.215172: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-21 22:10:32.215684: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-21 22:10:32.218645: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-21 22:10:32.219726: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-21 22:10:32.224506: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-21 22:10:32.226026: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-21 22:10:32.226110: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-21 22:10:32.226909: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-08-21 22:10:32.226925: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-08-21 22:10:32.226934: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-08-21 22:10:32.228332: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9833 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-08-21 22:10:32.488458: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-21 22:10:32.488537: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-21 22:10:32.488560: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-21 22:10:32.488579: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-21 22:10:32.488597: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-21 22:10:32.488615: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-21 22:10:32.488633: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-21 22:10:32.488651: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-21 22:10:32.490200: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-21 22:10:32.491506: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-21 22:10:32.491541: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-21 22:10:32.491560: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-21 22:10:32.491578: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-21 22:10:32.491595: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-21 22:10:32.491612: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-21 22:10:32.491630: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-21 22:10:32.491647: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-21 22:10:32.493207: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-21 22:10:32.493239: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-08-21 22:10:32.493249: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-08-21 22:10:32.493258: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-08-21 22:10:32.494852: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9833 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-08-21 22:10:41.276963: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-21 22:10:41.468210: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 16 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 34.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 25.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 28.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 27.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 27.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 28.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 33.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 2 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 31.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 31.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 2 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 29.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 28.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 25.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 28.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 32.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 28.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 31.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 Detection mask done ! Trying to reset tf kernel 1282937 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 747 tf kernel not reseted sub process len(results) : 16 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 16 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 6036 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.002176523208618164 nb_pixel_total : 99213 time to create 1 rle with old method : 0.11709189414978027 length of segment : 902 time for calcul the mask position with numpy : 0.001491546630859375 nb_pixel_total : 71008 time to create 1 rle with old method : 0.08269333839416504 length of segment : 517 time for calcul the mask position with numpy : 0.014382362365722656 nb_pixel_total : 737685 time to create 1 rle with new method : 0.03892087936401367 length of segment : 926 time for calcul the mask position with numpy : 0.00014710426330566406 nb_pixel_total : 6857 time to create 1 rle with old method : 0.008636474609375 length of segment : 45 time for calcul the mask position with numpy : 0.00016045570373535156 nb_pixel_total : 8069 time to create 1 rle with old method : 0.009624719619750977 length of segment : 119 time for calcul the mask position with numpy : 0.00031065940856933594 nb_pixel_total : 18791 time to create 1 rle with old method : 0.021547794342041016 length of segment : 215 time for calcul the mask position with numpy : 0.00043892860412597656 nb_pixel_total : 30588 time to create 1 rle with old method : 0.03474760055541992 length of segment : 279 time for calcul the mask position with numpy : 0.001293182373046875 nb_pixel_total : 97236 time to create 1 rle with old method : 0.11006021499633789 length of segment : 503 time for calcul the mask position with numpy : 0.00015926361083984375 nb_pixel_total : 7524 time to create 1 rle with old method : 0.008615732192993164 length of segment : 113 time for calcul the mask position with numpy : 0.00023698806762695312 nb_pixel_total : 10654 time to create 1 rle with old method : 0.012239694595336914 length of segment : 183 time for calcul the mask position with numpy : 9.059906005859375e-05 nb_pixel_total : 2767 time to create 1 rle with old method : 0.0033652782440185547 length of segment : 58 time for calcul the mask position with numpy : 0.01654052734375 nb_pixel_total : 737312 time to create 1 rle with new method : 1.0263988971710205 length of segment : 981 time for calcul the mask position with numpy : 0.0005605220794677734 nb_pixel_total : 17081 time to create 1 rle with old method : 0.019999027252197266 length of segment : 168 time for calcul the mask position with numpy : 0.010906457901000977 nb_pixel_total : 613644 time to create 1 rle with new method : 0.036179542541503906 length of segment : 1172 time for calcul the mask position with numpy : 0.0008013248443603516 nb_pixel_total : 26367 time to create 1 rle with old method : 0.030506610870361328 length of segment : 313 time for calcul the mask position with numpy : 0.0029926300048828125 nb_pixel_total : 59694 time to create 1 rle with old method : 0.07817387580871582 length of segment : 915 time for calcul the mask position with numpy : 0.0026628971099853516 nb_pixel_total : 121318 time to create 1 rle with old method : 0.1459484100341797 length of segment : 563 time for calcul the mask position with numpy : 0.002379179000854492 nb_pixel_total : 112640 time to create 1 rle with old method : 0.13440418243408203 length of segment : 547 time for calcul the mask position with numpy : 0.010637760162353516 nb_pixel_total : 621362 time to create 1 rle with new method : 0.036743879318237305 length of segment : 887 time for calcul the mask position with numpy : 0.0003724098205566406 nb_pixel_total : 8439 time to create 1 rle with old method : 0.010426759719848633 length of segment : 123 time for calcul the mask position with numpy : 0.010075092315673828 nb_pixel_total : 618855 time to create 1 rle with new method : 0.01591777801513672 length of segment : 896 time for calcul the mask position with numpy : 0.019115686416625977 nb_pixel_total : 1017171 time to create 1 rle with new method : 0.06866288185119629 length of segment : 1686 time for calcul the mask position with numpy : 0.0016880035400390625 nb_pixel_total : 108248 time to create 1 rle with old method : 0.1414778232574463 length of segment : 552 time for calcul the mask position with numpy : 0.0002567768096923828 nb_pixel_total : 4855 time to create 1 rle with old method : 0.005870819091796875 length of segment : 104 time for calcul the mask position with numpy : 0.00044465065002441406 nb_pixel_total : 17515 time to create 1 rle with old method : 0.020883560180664062 length of segment : 156 time for calcul the mask position with numpy : 0.0008378028869628906 nb_pixel_total : 21501 time to create 1 rle with old method : 0.04197192192077637 length of segment : 154 time for calcul the mask position with numpy : 0.01276087760925293 nb_pixel_total : 647227 time to create 1 rle with new method : 0.09305548667907715 length of segment : 1161 time for calcul the mask position with numpy : 0.00037288665771484375 nb_pixel_total : 10075 time to create 1 rle with old method : 0.012396812438964844 length of segment : 181 time for calcul the mask position with numpy : 0.002301931381225586 nb_pixel_total : 113660 time to create 1 rle with old method : 0.1426098346710205 length of segment : 585 time spent for convertir_results : 4.886908054351807 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 29 chid ids of type : 3594 Number RLEs to save : 15004 save missing photos in datou_result : time spend for datou_step_exec : 28.595577478408813 time spend to save output : 0.9360456466674805 total time spend for step 1 : 29.531623125076294 step2:crop_condition Thu Aug 21 22:10:58 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 16 ! batch 1 Loaded 29 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 3736932 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1755807060_1282424 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 5 photos in the portfolio 3736932 time of upload the photos Elapsed time : 2.5932669639587402 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 3736932 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1755807068_1282424 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 5 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.2391252517700195 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 19 About to insert : list_path_to_insert length 19 new photo from crops ! About to upload 19 photos upload in portfolio : 3736932 init cache_photo without model_param we have 19 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1755807088_1282424 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 19 photos in the portfolio 3736932 time of upload the photos Elapsed time : 4.89072060585022 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1378209803, 1378209801, 1378209799, 1378209797, 1378209795, 1378209787, 1378209784, 1378209783, 1378209782, 1378209780, 1378209779, 1378209762, 1378209761, 1378209760, 1378209759, 1378209757] Looping around the photos to save general results len do output : 29 /1378270388Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378270389Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378270391Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378270392Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378270393Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378270451Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378270453Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378270455Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378270457Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378270459Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378270780Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378270781Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378270783Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378270785Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378270786Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378270787Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378270790Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378270791Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378270793Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378270796Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378270798Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378270799Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378270801Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378270804Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378270805Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378270807Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378270809Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378270811Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378270813Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209803', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209801', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209799', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209797', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209795', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209787', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209784', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209783', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209782', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209780', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209779', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209762', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209761', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209760', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209759', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209757', None, None, None, None, None, '3568988') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 103 time used for this insertion : 0.037750959396362305 save_final save missing photos in datou_result : time spend for datou_step_exec : 33.860267639160156 time spend to save output : 0.03912520408630371 total time spend for step 2 : 33.89939284324646 step3:rle_unique_nms_with_priority Thu Aug 21 22:11:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 29 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.13687562942504883 time for calcul the mask position with numpy : 0.06086254119873047 nb_pixel_total : 1974387 time to create 1 rle with new method : 0.17277860641479492 time for calcul the mask position with numpy : 0.006772518157958984 nb_pixel_total : 99213 time to create 1 rle with old method : 0.11456966400146484 create new chi : 0.366410493850708 time to delete rle : 0.028841733932495117 batch 1 Loaded 3 chid ids of type : 3594 +++++Number RLEs to save : 2884 TO DO : save crop sub photo not yet done ! save time : 0.2621290683746338 No data in photo_id : 1378209801 nb_obj : 2 nb_hashtags : 1 time to prepare the origin masks : 0.40067505836486816 time for calcul the mask position with numpy : 0.03840231895446777 nb_pixel_total : 1264907 time to create 1 rle with new method : 0.26009559631347656 time for calcul the mask position with numpy : 0.01140737533569336 nb_pixel_total : 737685 time to create 1 rle with new method : 0.07276535034179688 time for calcul the mask position with numpy : 0.006802082061767578 nb_pixel_total : 71008 time to create 1 rle with old method : 0.08658099174499512 create new chi : 0.4849874973297119 time to delete rle : 0.0007181167602539062 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 3966 TO DO : save crop sub photo not yet done ! save time : 0.31078028678894043 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.05892467498779297 time for calcul the mask position with numpy : 0.061737060546875 nb_pixel_total : 2039883 time to create 1 rle with new method : 0.19045543670654297 time for calcul the mask position with numpy : 0.0066602230072021484 nb_pixel_total : 18791 time to create 1 rle with old method : 0.021668195724487305 time for calcul the mask position with numpy : 0.006109476089477539 nb_pixel_total : 8069 time to create 1 rle with old method : 0.009350299835205078 time for calcul the mask position with numpy : 0.007120847702026367 nb_pixel_total : 6857 time to create 1 rle with old method : 0.008272886276245117 create new chi : 0.3212621212005615 time to delete rle : 0.0003180503845214844 batch 1 Loaded 7 chid ids of type : 3594 +++Number RLEs to save : 1838 TO DO : save crop sub photo not yet done ! save time : 0.14415955543518066 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.054254770278930664 time for calcul the mask position with numpy : 0.12134718894958496 nb_pixel_total : 1938252 time to create 1 rle with new method : 0.08111929893493652 time for calcul the mask position with numpy : 0.006270647048950195 nb_pixel_total : 7524 time to create 1 rle with old method : 0.008836507797241211 time for calcul the mask position with numpy : 0.006517648696899414 nb_pixel_total : 97236 time to create 1 rle with old method : 0.11229538917541504 time for calcul the mask position with numpy : 0.0062944889068603516 nb_pixel_total : 30588 time to create 1 rle with old method : 0.03513073921203613 create new chi : 0.38986849784851074 time to delete rle : 0.0002849102020263672 batch 1 Loaded 7 chid ids of type : 3594 +++Number RLEs to save : 2870 TO DO : save crop sub photo not yet done ! save time : 0.20681142807006836 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.054459571838378906 time for calcul the mask position with numpy : 0.18418407440185547 nb_pixel_total : 1324461 time to create 1 rle with new method : 0.07916545867919922 time for calcul the mask position with numpy : 0.01107335090637207 nb_pixel_total : 735718 time to create 1 rle with new method : 0.14881110191345215 time for calcul the mask position with numpy : 0.00764775276184082 nb_pixel_total : 2767 time to create 1 rle with old method : 0.004143714904785156 time for calcul the mask position with numpy : 0.007298946380615234 nb_pixel_total : 10654 time to create 1 rle with old method : 0.01268911361694336 create new chi : 0.46489477157592773 time to delete rle : 0.0005550384521484375 batch 1 Loaded 7 chid ids of type : 3594 +++Number RLEs to save : 3439 TO DO : save crop sub photo not yet done ! save time : 0.2676723003387451 No data in photo_id : 1378209784 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.043157339096069336 time for calcul the mask position with numpy : 0.020642757415771484 nb_pixel_total : 2056519 time to create 1 rle with new method : 0.08981585502624512 time for calcul the mask position with numpy : 0.006739139556884766 nb_pixel_total : 17081 time to create 1 rle with old method : 0.02852320671081543 create new chi : 0.15901446342468262 time to delete rle : 0.0003077983856201172 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 1416 TO DO : save crop sub photo not yet done ! save time : 0.11587882041931152 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.036646127700805664 time for calcul the mask position with numpy : 0.08835911750793457 nb_pixel_total : 1459956 time to create 1 rle with new method : 0.08037567138671875 time for calcul the mask position with numpy : 0.012157440185546875 nb_pixel_total : 613644 time to create 1 rle with new method : 0.17151141166687012 create new chi : 0.3631017208099365 time to delete rle : 0.00032067298889160156 batch 1 Loaded 3 chid ids of type : 3594 ++++Number RLEs to save : 3424 TO DO : save crop sub photo not yet done ! save time : 0.22620058059692383 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.2216019630432129 time for calcul the mask position with numpy : 0.05829954147338867 nb_pixel_total : 1987539 time to create 1 rle with new method : 0.13464736938476562 time for calcul the mask position with numpy : 0.0064966678619384766 nb_pixel_total : 59694 time to create 1 rle with old method : 0.07894158363342285 time for calcul the mask position with numpy : 0.0065762996673583984 nb_pixel_total : 26367 time to create 1 rle with old method : 0.03140377998352051 create new chi : 0.3273894786834717 time to delete rle : 0.0008180141448974609 batch 1 Loaded 5 chid ids of type : 3594 ++++++Number RLEs to save : 3536 TO DO : save crop sub photo not yet done ! save time : 0.2588047981262207 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.03897547721862793 time for calcul the mask position with numpy : 0.04092764854431152 nb_pixel_total : 1952282 time to create 1 rle with new method : 0.09082722663879395 time for calcul the mask position with numpy : 0.006985902786254883 nb_pixel_total : 121318 time to create 1 rle with old method : 0.1510603427886963 create new chi : 0.29915714263916016 time to delete rle : 0.00043392181396484375 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 2206 TO DO : save crop sub photo not yet done ! save time : 0.16974282264709473 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.11979055404663086 time for calcul the mask position with numpy : 0.038558244705200195 nb_pixel_total : 1339598 time to create 1 rle with new method : 0.1745600700378418 time for calcul the mask position with numpy : 0.010032176971435547 nb_pixel_total : 621362 time to create 1 rle with new method : 0.07282257080078125 time for calcul the mask position with numpy : 0.006741046905517578 nb_pixel_total : 112640 time to create 1 rle with old method : 0.13110756874084473 create new chi : 0.44331789016723633 time to delete rle : 0.0006971359252929688 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 3948 TO DO : save crop sub photo not yet done ! save time : 0.24405384063720703 nb_obj : 2 nb_hashtags : 1 time to prepare the origin masks : 0.05822038650512695 time for calcul the mask position with numpy : 0.09423947334289551 nb_pixel_total : 1446306 time to create 1 rle with new method : 0.15169835090637207 time for calcul the mask position with numpy : 0.009967327117919922 nb_pixel_total : 618855 time to create 1 rle with new method : 0.07383084297180176 time for calcul the mask position with numpy : 0.00719451904296875 nb_pixel_total : 8439 time to create 1 rle with old method : 0.017337560653686523 create new chi : 0.3714635372161865 time to delete rle : 0.0009715557098388672 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 3118 TO DO : save crop sub photo not yet done ! save time : 0.2612602710723877 nb_obj : 2 nb_hashtags : 1 time to prepare the origin masks : 0.05686497688293457 time for calcul the mask position with numpy : 0.03537583351135254 nb_pixel_total : 948181 time to create 1 rle with new method : 0.0989382266998291 time for calcul the mask position with numpy : 0.007054328918457031 nb_pixel_total : 108248 time to create 1 rle with old method : 0.15594029426574707 time for calcul the mask position with numpy : 0.01324009895324707 nb_pixel_total : 1017171 time to create 1 rle with new method : 0.1626143455505371 create new chi : 0.49085140228271484 time to delete rle : 0.0008058547973632812 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 5556 TO DO : save crop sub photo not yet done ! save time : 0.43060946464538574 nb_obj : 5 nb_hashtags : 3 time to prepare the origin masks : 0.20148229598999023 time for calcul the mask position with numpy : 0.17784523963928223 nb_pixel_total : 1372427 time to create 1 rle with new method : 0.1878371238708496 time for calcul the mask position with numpy : 0.00696563720703125 nb_pixel_total : 10075 time to create 1 rle with old method : 0.014354228973388672 time for calcul the mask position with numpy : 0.01042318344116211 nb_pixel_total : 647227 time to create 1 rle with new method : 0.08099079132080078 time for calcul the mask position with numpy : 0.006539106369018555 nb_pixel_total : 21501 time to create 1 rle with old method : 0.026914358139038086 time for calcul the mask position with numpy : 0.007677555084228516 nb_pixel_total : 17515 time to create 1 rle with old method : 0.027671098709106445 time for calcul the mask position with numpy : 0.0066149234771728516 nb_pixel_total : 4855 time to create 1 rle with old method : 0.005784034729003906 create new chi : 0.5688745975494385 time to delete rle : 0.0008549690246582031 batch 1 Loaded 11 chid ids of type : 3594 ++++++++Number RLEs to save : 4592 TO DO : save crop sub photo not yet done ! save time : 0.31930017471313477 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.046674489974975586 time for calcul the mask position with numpy : 0.021687984466552734 nb_pixel_total : 1959940 time to create 1 rle with new method : 0.047109365463256836 time for calcul the mask position with numpy : 0.00841379165649414 nb_pixel_total : 113660 time to create 1 rle with old method : 0.13219261169433594 create new chi : 0.20981121063232422 time to delete rle : 0.000347137451171875 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 2250 TO DO : save crop sub photo not yet done ! save time : 0.17732977867126465 map_output_result : {1378209803: (0.0, 'Should be the crop_list due to order', 0), 1378209801: (0.0, 'Should be the crop_list due to order', 0.0), 1378209799: (0.0, 'Should be the crop_list due to order', 0), 1378209797: (0.0, 'Should be the crop_list due to order', 0), 1378209795: (0.0, 'Should be the crop_list due to order', 0), 1378209787: (0.0, 'Should be the crop_list due to order', 0), 1378209784: (0.0, 'Should be the crop_list due to order', 0.0), 1378209783: (0.0, 'Should be the crop_list due to order', 0), 1378209782: (0.0, 'Should be the crop_list due to order', 0), 1378209780: (0.0, 'Should be the crop_list due to order', 0), 1378209779: (0.0, 'Should be the crop_list due to order', 0), 1378209762: (0.0, 'Should be the crop_list due to order', 0), 1378209761: (0.0, 'Should be the crop_list due to order', 0), 1378209760: (0.0, 'Should be the crop_list due to order', 0), 1378209759: (0.0, 'Should be the crop_list due to order', 0), 1378209757: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1378209803, 1378209801, 1378209799, 1378209797, 1378209795, 1378209787, 1378209784, 1378209783, 1378209782, 1378209780, 1378209779, 1378209762, 1378209761, 1378209760, 1378209759, 1378209757] Looping around the photos to save general results len do output : 16 /1378209803.Didn't retrieve data . /1378209801.Didn't retrieve data . /1378209799.Didn't retrieve data . /1378209797.Didn't retrieve data . /1378209795.Didn't retrieve data . /1378209787.Didn't retrieve data . /1378209784.Didn't retrieve data . /1378209783.Didn't retrieve data . /1378209782.Didn't retrieve data . /1378209780.Didn't retrieve data . /1378209779.Didn't retrieve data . /1378209762.Didn't retrieve data . /1378209761.Didn't retrieve data . /1378209760.Didn't retrieve data . /1378209759.Didn't retrieve data . /1378209757.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209803', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209801', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209799', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209797', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209795', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209787', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209784', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209783', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209782', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209780', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209779', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209762', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209761', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209760', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209759', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209757', None, None, None, None, None, '3568988') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 48 time used for this insertion : 0.014386653900146484 save_final save missing photos in datou_result : time spend for datou_step_exec : 10.777439594268799 time spend to save output : 0.015012264251708984 total time spend for step 3 : 10.792451858520508 step4:ventilate_hashtags_in_portfolio Thu Aug 21 22:11:43 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 26121460 get user id for portfolio 26121460 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26121460 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pehd','metal','environnement','carton','papier','flou','pet_clair','autre','mal_croppe','pet_fonce','background')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26121460 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pehd','metal','environnement','carton','papier','flou','pet_clair','autre','mal_croppe','pet_fonce','background')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26121460 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pehd','metal','environnement','carton','papier','flou','pet_clair','autre','mal_croppe','pet_fonce','background')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/26121653,26121654,26121655,26121656,26121657,26121658,26121659,26121660,26121661,26121662,26121663?tags=pehd,metal,environnement,carton,papier,flou,pet_clair,autre,mal_croppe,pet_fonce,background Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1378209803, 1378209801, 1378209799, 1378209797, 1378209795, 1378209787, 1378209784, 1378209783, 1378209782, 1378209780, 1378209779, 1378209762, 1378209761, 1378209760, 1378209759, 1378209757] Looping around the photos to save general results len do output : 1 /26121460. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209803', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209801', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209799', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209797', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209795', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209787', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209784', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209783', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209782', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209780', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209779', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209762', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209761', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209760', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209759', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209757', None, None, None, None, None, '3568988') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 17 time used for this insertion : 0.014925956726074219 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.6501185894012451 time spend to save output : 0.015273332595825195 total time spend for step 4 : 1.6653919219970703 step5:final Thu Aug 21 22:11:45 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1378209803: ('0.17981294608410492',), 1378209801: ('0.17981294608410492',), 1378209799: ('0.17981294608410492',), 1378209797: ('0.17981294608410492',), 1378209795: ('0.17981294608410492',), 1378209787: ('0.17981294608410492',), 1378209784: ('0.17981294608410492',), 1378209783: ('0.17981294608410492',), 1378209782: ('0.17981294608410492',), 1378209780: ('0.17981294608410492',), 1378209779: ('0.17981294608410492',), 1378209762: ('0.17981294608410492',), 1378209761: ('0.17981294608410492',), 1378209760: ('0.17981294608410492',), 1378209759: ('0.17981294608410492',), 1378209757: ('0.17981294608410492',)} new output for save of step final : {1378209803: ('0.17981294608410492',), 1378209801: ('0.17981294608410492',), 1378209799: ('0.17981294608410492',), 1378209797: ('0.17981294608410492',), 1378209795: ('0.17981294608410492',), 1378209787: ('0.17981294608410492',), 1378209784: ('0.17981294608410492',), 1378209783: ('0.17981294608410492',), 1378209782: ('0.17981294608410492',), 1378209780: ('0.17981294608410492',), 1378209779: ('0.17981294608410492',), 1378209762: ('0.17981294608410492',), 1378209761: ('0.17981294608410492',), 1378209760: ('0.17981294608410492',), 1378209759: ('0.17981294608410492',), 1378209757: ('0.17981294608410492',)} [1378209803, 1378209801, 1378209799, 1378209797, 1378209795, 1378209787, 1378209784, 1378209783, 1378209782, 1378209780, 1378209779, 1378209762, 1378209761, 1378209760, 1378209759, 1378209757] Looping around the photos to save general results len do output : 16 /1378209803.Didn't retrieve data . /1378209801.Didn't retrieve data . /1378209799.Didn't retrieve data . /1378209797.Didn't retrieve data . /1378209795.Didn't retrieve data . /1378209787.Didn't retrieve data . /1378209784.Didn't retrieve data . /1378209783.Didn't retrieve data . /1378209782.Didn't retrieve data . /1378209780.Didn't retrieve data . /1378209779.Didn't retrieve data . /1378209762.Didn't retrieve data . /1378209761.Didn't retrieve data . /1378209760.Didn't retrieve data . /1378209759.Didn't retrieve data . /1378209757.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209803', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209801', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209799', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209797', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209795', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209787', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209784', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209783', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209782', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209780', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209779', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209762', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209761', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209760', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209759', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209757', None, None, None, None, None, '3568988') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 48 time used for this insertion : 0.01413106918334961 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.12199187278747559 time spend to save output : 0.014864921569824219 total time spend for step 5 : 0.1368567943572998 step6:blur_detection Thu Aug 21 22:11:45 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1755807027_1282424_1378209803_026eeafe47915925a92318fee18e9f89.jpg resize: (1080, 1920) 1378209803 -2.5407248868906627 treat image : temp/1755807027_1282424_1378209801_639bb1c75f1e343d470391d840788609.jpg resize: (1080, 1920) 1378209801 -3.0546488746896054 treat image : temp/1755807027_1282424_1378209799_b30f972597be1e1f23a2a873ed468db4.jpg resize: (1080, 1920) 1378209799 -2.9835091602765798 treat image : temp/1755807027_1282424_1378209797_fabaae6ca165ec205648fc4be741e33f.jpg resize: (1080, 1920) 1378209797 -1.2278151318243622 treat image : temp/1755807027_1282424_1378209795_96c781394d22c6c61d34a8d00e351916.jpg resize: (1080, 1920) 1378209795 -1.8972449860106462 treat image : temp/1755807027_1282424_1378209787_9cc992ad2e55e04aeaa3b354205aec17.jpg resize: (1080, 1920) 1378209787 -2.4932140366263367 treat image : temp/1755807027_1282424_1378209784_0804232786909c92a6bac760cc36870f.jpg resize: (1080, 1920) 1378209784 -1.6501957721927398 treat image : temp/1755807027_1282424_1378209783_3c857448173f31d586acbff92ed99be8.jpg resize: (1080, 1920) 1378209783 -1.750304731844094 treat image : temp/1755807027_1282424_1378209782_10a114160fef5f5f8cbbb5f457a7526e.jpg resize: (1080, 1920) 1378209782 -2.458268192198136 treat image : temp/1755807027_1282424_1378209780_746c16f051fd54af17dfc12961698a0f.jpg resize: (1080, 1920) 1378209780 -2.4003322253033836 treat image : temp/1755807027_1282424_1378209779_9bf87d7b659373f265a5942997d7c5ba.jpg resize: (1080, 1920) 1378209779 -2.6803290816808727 treat image : temp/1755807027_1282424_1378209762_10e6bcc60aa167593fb3ae31152b85aa.jpg resize: (1080, 1920) 1378209762 -2.992152017941708 treat image : temp/1755807027_1282424_1378209761_6c34eb07a4d56bf872d838877c9bda39.jpg resize: (1080, 1920) 1378209761 -2.6371248335623085 treat image : temp/1755807027_1282424_1378209760_f67cfee553ce4dc62e793185aa55c688.jpg resize: (1080, 1920) 1378209760 -2.5897079114829245 treat image : temp/1755807027_1282424_1378209759_e1093a85753e160313710fb518943f2a.jpg resize: (1080, 1920) 1378209759 -1.5257710166870098 treat image : temp/1755807027_1282424_1378209757_37fc692c799b8d340ea334f19ff6023d.jpg resize: (1080, 1920) 1378209757 -2.474659859684509 treat image : temp/1755807027_1282424_1378209797_fabaae6ca165ec205648fc4be741e33f_rle_crop_3923259018_0.png resize: (119, 108) 1378270388 -1.8081723763978386 treat image : temp/1755807027_1282424_1378209795_96c781394d22c6c61d34a8d00e351916_rle_crop_3923259022_0.png resize: (113, 98) 1378270389 -1.4977329096095382 treat image : temp/1755807027_1282424_1378209787_9cc992ad2e55e04aeaa3b354205aec17_rle_crop_3923259023_0.png resize: (182, 106) 1378270391 -0.5571996984392934 treat image : temp/1755807027_1282424_1378209787_9cc992ad2e55e04aeaa3b354205aec17_rle_crop_3923259024_0.png resize: (58, 70) 1378270392 0.13640826606574372 treat image : temp/1755807027_1282424_1378209759_e1093a85753e160313710fb518943f2a_rle_crop_3923259041_0.png resize: (181, 107) 1378270393 -1.1175876775626263 treat image : temp/1755807027_1282424_1378209780_746c16f051fd54af17dfc12961698a0f_rle_crop_3923259028_0.png resize: (284, 139) 1378270451 -1.302838318525739 treat image : temp/1755807027_1282424_1378209762_10e6bcc60aa167593fb3ae31152b85aa_rle_crop_3923259032_0.png resize: (887, 971) 1378270453 -0.283525671626892 treat image : temp/1755807027_1282424_1378209761_6c34eb07a4d56bf872d838877c9bda39_rle_crop_3923259034_0.png resize: (895, 954) 1378270455 -0.0856088226078055 treat image : temp/1755807027_1282424_1378209761_6c34eb07a4d56bf872d838877c9bda39_rle_crop_3923259033_0.png resize: (123, 112) 1378270457 -1.816326874539863 treat image : temp/1755807027_1282424_1378209759_e1093a85753e160313710fb518943f2a_rle_crop_3923259037_0.png resize: (104, 67) 1378270459 -0.9884931346278354 treat image : temp/1755807027_1282424_1378209803_026eeafe47915925a92318fee18e9f89_rle_crop_3923259014_0.png resize: (531, 341) 1378270780 -1.5953007558142254 treat image : temp/1755807027_1282424_1378209799_b30f972597be1e1f23a2a873ed468db4_rle_crop_3923259016_0.png resize: (925, 1009) 1378270781 -0.032599165642513604 treat image : temp/1755807027_1282424_1378209799_b30f972597be1e1f23a2a873ed468db4_rle_crop_3923259015_0.png resize: (510, 321) 1378270783 -0.9215783896182036 treat image : temp/1755807027_1282424_1378209797_fabaae6ca165ec205648fc4be741e33f_rle_crop_3923259017_0.png resize: (40, 197) 1378270785 -3.2585813980434253 treat image : temp/1755807027_1282424_1378209797_fabaae6ca165ec205648fc4be741e33f_rle_crop_3923259019_0.png resize: (215, 147) 1378270786 -1.712556070567335 treat image : temp/1755807027_1282424_1378209795_96c781394d22c6c61d34a8d00e351916_rle_crop_3923259021_0.png resize: (492, 345) 1378270787 -0.5096449126099855 treat image : temp/1755807027_1282424_1378209795_96c781394d22c6c61d34a8d00e351916_rle_crop_3923259020_0.png resize: (279, 174) 1378270790 -1.58910996760939 treat image : temp/1755807027_1282424_1378209787_9cc992ad2e55e04aeaa3b354205aec17_rle_crop_3923259025_0.png resize: (974, 1019) 1378270791 0.30531309756148683 treat image : temp/1755807027_1282424_1378209783_3c857448173f31d586acbff92ed99be8_rle_crop_3923259026_0.png resize: (168, 142) 1378270793 -3.816766528943857 treat image : temp/1755807027_1282424_1378209782_10a114160fef5f5f8cbbb5f457a7526e_rle_crop_3923259027_0.png resize: (899, 850) 1378270796 0.0610078470134385 treat image : temp/1755807027_1282424_1378209780_746c16f051fd54af17dfc12961698a0f_rle_crop_3923259029_0.png resize: (537, 359) 1378270798 -2.144350196347839 treat image : temp/1755807027_1282424_1378209779_9bf87d7b659373f265a5942997d7c5ba_rle_crop_3923259030_0.png resize: (544, 361) 1378270799 -0.20403465242195018 treat image : temp/1755807027_1282424_1378209762_10e6bcc60aa167593fb3ae31152b85aa_rle_crop_3923259031_0.png resize: (532, 343) 1378270801 -0.1394535914649813 treat image : temp/1755807027_1282424_1378209760_f67cfee553ce4dc62e793185aa55c688_rle_crop_3923259035_0.png resize: (991, 1307) 1378270804 -2.1613354278642656 treat image : temp/1755807027_1282424_1378209760_f67cfee553ce4dc62e793185aa55c688_rle_crop_3923259036_0.png resize: (542, 345) 1378270805 -0.717058884523761 treat image : temp/1755807027_1282424_1378209759_e1093a85753e160313710fb518943f2a_rle_crop_3923259040_0.png resize: (912, 921) 1378270807 0.12480758475692919 treat image : temp/1755807027_1282424_1378209759_e1093a85753e160313710fb518943f2a_rle_crop_3923259038_0.png resize: (156, 149) 1378270809 -2.632250443964107 treat image : temp/1755807027_1282424_1378209759_e1093a85753e160313710fb518943f2a_rle_crop_3923259039_0.png resize: (154, 182) 1378270811 -4.541031134530276 treat image : temp/1755807027_1282424_1378209757_37fc692c799b8d340ea334f19ff6023d_rle_crop_3923259042_0.png resize: (570, 343) 1378270813 -0.09614582079618504 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 45 time used for this insertion : 0.013257503509521484 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 45 time used for this insertion : 0.012887954711914062 save missing photos in datou_result : time spend for datou_step_exec : 16.182679891586304 time spend to save output : 0.031134366989135742 total time spend for step 6 : 16.21381425857544 step7:brightness Thu Aug 21 22:12:01 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1755807027_1282424_1378209803_026eeafe47915925a92318fee18e9f89.jpg treat image : temp/1755807027_1282424_1378209801_639bb1c75f1e343d470391d840788609.jpg treat image : temp/1755807027_1282424_1378209799_b30f972597be1e1f23a2a873ed468db4.jpg treat image : temp/1755807027_1282424_1378209797_fabaae6ca165ec205648fc4be741e33f.jpg treat image : temp/1755807027_1282424_1378209795_96c781394d22c6c61d34a8d00e351916.jpg treat image : temp/1755807027_1282424_1378209787_9cc992ad2e55e04aeaa3b354205aec17.jpg treat image : temp/1755807027_1282424_1378209784_0804232786909c92a6bac760cc36870f.jpg treat image : temp/1755807027_1282424_1378209783_3c857448173f31d586acbff92ed99be8.jpg treat image : temp/1755807027_1282424_1378209782_10a114160fef5f5f8cbbb5f457a7526e.jpg treat image : temp/1755807027_1282424_1378209780_746c16f051fd54af17dfc12961698a0f.jpg treat image : temp/1755807027_1282424_1378209779_9bf87d7b659373f265a5942997d7c5ba.jpg treat image : temp/1755807027_1282424_1378209762_10e6bcc60aa167593fb3ae31152b85aa.jpg treat image : temp/1755807027_1282424_1378209761_6c34eb07a4d56bf872d838877c9bda39.jpg treat image : temp/1755807027_1282424_1378209760_f67cfee553ce4dc62e793185aa55c688.jpg treat image : temp/1755807027_1282424_1378209759_e1093a85753e160313710fb518943f2a.jpg treat image : temp/1755807027_1282424_1378209757_37fc692c799b8d340ea334f19ff6023d.jpg treat image : temp/1755807027_1282424_1378209797_fabaae6ca165ec205648fc4be741e33f_rle_crop_3923259018_0.png treat image : temp/1755807027_1282424_1378209795_96c781394d22c6c61d34a8d00e351916_rle_crop_3923259022_0.png treat image : temp/1755807027_1282424_1378209787_9cc992ad2e55e04aeaa3b354205aec17_rle_crop_3923259023_0.png treat image : temp/1755807027_1282424_1378209787_9cc992ad2e55e04aeaa3b354205aec17_rle_crop_3923259024_0.png treat image : temp/1755807027_1282424_1378209759_e1093a85753e160313710fb518943f2a_rle_crop_3923259041_0.png treat image : temp/1755807027_1282424_1378209780_746c16f051fd54af17dfc12961698a0f_rle_crop_3923259028_0.png treat image : temp/1755807027_1282424_1378209762_10e6bcc60aa167593fb3ae31152b85aa_rle_crop_3923259032_0.png treat image : temp/1755807027_1282424_1378209761_6c34eb07a4d56bf872d838877c9bda39_rle_crop_3923259034_0.png treat image : temp/1755807027_1282424_1378209761_6c34eb07a4d56bf872d838877c9bda39_rle_crop_3923259033_0.png treat image : temp/1755807027_1282424_1378209759_e1093a85753e160313710fb518943f2a_rle_crop_3923259037_0.png treat image : temp/1755807027_1282424_1378209803_026eeafe47915925a92318fee18e9f89_rle_crop_3923259014_0.png treat image : temp/1755807027_1282424_1378209799_b30f972597be1e1f23a2a873ed468db4_rle_crop_3923259016_0.png treat image : temp/1755807027_1282424_1378209799_b30f972597be1e1f23a2a873ed468db4_rle_crop_3923259015_0.png treat image : temp/1755807027_1282424_1378209797_fabaae6ca165ec205648fc4be741e33f_rle_crop_3923259017_0.png treat image : temp/1755807027_1282424_1378209797_fabaae6ca165ec205648fc4be741e33f_rle_crop_3923259019_0.png treat image : temp/1755807027_1282424_1378209795_96c781394d22c6c61d34a8d00e351916_rle_crop_3923259021_0.png treat image : temp/1755807027_1282424_1378209795_96c781394d22c6c61d34a8d00e351916_rle_crop_3923259020_0.png treat image : temp/1755807027_1282424_1378209787_9cc992ad2e55e04aeaa3b354205aec17_rle_crop_3923259025_0.png treat image : temp/1755807027_1282424_1378209783_3c857448173f31d586acbff92ed99be8_rle_crop_3923259026_0.png treat image : temp/1755807027_1282424_1378209782_10a114160fef5f5f8cbbb5f457a7526e_rle_crop_3923259027_0.png treat image : temp/1755807027_1282424_1378209780_746c16f051fd54af17dfc12961698a0f_rle_crop_3923259029_0.png treat image : temp/1755807027_1282424_1378209779_9bf87d7b659373f265a5942997d7c5ba_rle_crop_3923259030_0.png treat image : temp/1755807027_1282424_1378209762_10e6bcc60aa167593fb3ae31152b85aa_rle_crop_3923259031_0.png treat image : temp/1755807027_1282424_1378209760_f67cfee553ce4dc62e793185aa55c688_rle_crop_3923259035_0.png treat image : temp/1755807027_1282424_1378209760_f67cfee553ce4dc62e793185aa55c688_rle_crop_3923259036_0.png treat image : temp/1755807027_1282424_1378209759_e1093a85753e160313710fb518943f2a_rle_crop_3923259040_0.png treat image : temp/1755807027_1282424_1378209759_e1093a85753e160313710fb518943f2a_rle_crop_3923259038_0.png treat image : temp/1755807027_1282424_1378209759_e1093a85753e160313710fb518943f2a_rle_crop_3923259039_0.png treat image : temp/1755807027_1282424_1378209757_37fc692c799b8d340ea334f19ff6023d_rle_crop_3923259042_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 45 time used for this insertion : 0.013675689697265625 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 45 time used for this insertion : 0.014075517654418945 save missing photos in datou_result : time spend for datou_step_exec : 4.489961862564087 time spend to save output : 0.032105445861816406 total time spend for step 7 : 4.522067308425903 step8:velours_tree Thu Aug 21 22:12:06 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.09283280372619629 time spend to save output : 4.076957702636719e-05 total time spend for step 8 : 0.09287357330322266 step9:send_mail_cod Thu Aug 21 22:12:06 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P26121460_21-08-2025_22_12_06.pdf 26121653 imagette261216531755807126 26121654 imagette261216541755807126 26121656 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette261216561755807126 26121657 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette261216571755807126 26121658 imagette261216581755807126 26121659 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette261216591755807126 26121660 imagette261216601755807128 26121661 imagette261216611755807128 26121662 imagette261216621755807128 26121663 imagette261216631755807128 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=26121460 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/26121653,26121654,26121655,26121656,26121657,26121658,26121659,26121660,26121661,26121662,26121663?tags=pehd,metal,environnement,carton,papier,flou,pet_clair,autre,mal_croppe,pet_fonce,background args[1378209803] : ((1378209803, -2.5407248868906627, 492609224), (1378209803, 0.6292750759207764, 2107752395), '0.17981294608410492') We are sending mail with results at report@fotonower.com args[1378209801] : ((1378209801, -3.0546488746896054, 492609224), (1378209801, 0.7090642324124089, 2107752395), '0.17981294608410492') We are sending mail with results at report@fotonower.com args[1378209799] : ((1378209799, -2.9835091602765798, 492609224), (1378209799, 0.5558827236827665, 2107752395), '0.17981294608410492') We are sending mail with results at report@fotonower.com args[1378209797] : ((1378209797, -1.2278151318243622, 492688767), (1378209797, 0.48938988953341206, 2107752395), '0.17981294608410492') We are sending mail with results at report@fotonower.com args[1378209795] : ((1378209795, -1.8972449860106462, 492688767), (1378209795, 0.4273090221282327, 2107752395), '0.17981294608410492') We are sending mail with results at report@fotonower.com args[1378209787] : ((1378209787, -2.4932140366263367, 492609224), (1378209787, 0.5522683638289154, 2107752395), '0.17981294608410492') We are sending mail with results at report@fotonower.com args[1378209784] : ((1378209784, -1.6501957721927398, 492688767), (1378209784, 0.66357681837925, 2107752395), '0.17981294608410492') We are sending mail with results at report@fotonower.com args[1378209783] : ((1378209783, -1.750304731844094, 492688767), (1378209783, 0.6963903560716258, 2107752395), '0.17981294608410492') We are sending mail with results at report@fotonower.com args[1378209782] : ((1378209782, -2.458268192198136, 492609224), (1378209782, 0.534531971619708, 2107752395), '0.17981294608410492') We are sending mail with results at report@fotonower.com args[1378209780] : ((1378209780, -2.4003322253033836, 492609224), (1378209780, 0.48324028553147985, 2107752395), '0.17981294608410492') We are sending mail with results at report@fotonower.com args[1378209779] : ((1378209779, -2.6803290816808727, 492609224), (1378209779, 0.6537935131080332, 2107752395), '0.17981294608410492') We are sending mail with results at report@fotonower.com args[1378209762] : ((1378209762, -2.992152017941708, 492609224), (1378209762, 0.6967368396922071, 2107752395), '0.17981294608410492') We are sending mail with results at report@fotonower.com args[1378209761] : ((1378209761, -2.6371248335623085, 492609224), (1378209761, 0.4875775780272362, 2107752395), '0.17981294608410492') We are sending mail with results at report@fotonower.com args[1378209760] : ((1378209760, -2.5897079114829245, 492609224), (1378209760, 0.6191795216658762, 2107752395), '0.17981294608410492') We are sending mail with results at report@fotonower.com args[1378209759] : ((1378209759, -1.5257710166870098, 492688767), (1378209759, 0.6611995367388331, 2107752395), '0.17981294608410492') We are sending mail with results at report@fotonower.com args[1378209757] : ((1378209757, -2.474659859684509, 492609224), (1378209757, 0.6341726867960714, 2107752395), '0.17981294608410492') We are sending mail with results at report@fotonower.com refus_total : 0.17981294608410492 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=26121460 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26121460_21-08-2025_22_12_06.pdf results_Auto_P26121460_21-08-2025_22_12_06.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26121460_21-08-2025_22_12_06.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','26121460','results_Auto_P26121460_21-08-2025_22_12_06.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26121460_21-08-2025_22_12_06.pdf','pdf','','0.67','0.17981294608410492') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/26121460

https://www.fotonower.com/image?json=false&list_photos_id=1378209803
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378209801
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378209799
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378209797
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378209795
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378209787
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378209784
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378209783
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378209782
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378209780
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378209779
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378209762
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378209761
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378209760
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378209759
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378209757
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 17.98%
Veuillez trouver les photos des contaminants.

exemples de contaminants: carton: https://www.fotonower.com/view/26121656?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/26121657?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/26121659?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26121460_21-08-2025_22_12_06.pdf.

Lien vers velours :https://www.fotonower.com/velours/26121653,26121654,26121655,26121656,26121657,26121658,26121659,26121660,26121661,26121662,26121663?tags=pehd,metal,environnement,carton,papier,flou,pet_clair,autre,mal_croppe,pet_fonce,background.


L'équipe Fotonower 202 b'' Server: nginx Date: Thu, 21 Aug 2025 20:12:10 GMT Content-Length: 0 Connection: close X-Message-Id: mhGG1ZJlS7W0A25JT3Icjw Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1378209803, 1378209801, 1378209799, 1378209797, 1378209795, 1378209787, 1378209784, 1378209783, 1378209782, 1378209780, 1378209779, 1378209762, 1378209761, 1378209760, 1378209759, 1378209757] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209803', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209801', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209799', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209797', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209795', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209787', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209784', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209783', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209782', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209780', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209779', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209762', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209761', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209760', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209759', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209757', None, None, None, None, None, '3568988') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 16 time used for this insertion : 0.013074874877929688 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.6502838134765625 time spend to save output : 0.0133209228515625 total time spend for step 9 : 4.663604736328125 step10:split_time_score Thu Aug 21 22:12:10 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('12', 16),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 21082025 26121460 Nombre de photos uploadées : 16 / 23040 (0%) 21082025 26121460 Nombre de photos taguées (types de déchets): 0 / 16 (0%) 21082025 26121460 Nombre de photos taguées (volume) : 0 / 16 (0%) elapsed_time : load_data_split_time_score 6.198883056640625e-06 elapsed_time : order_list_meta_photo_and_scores 1.8596649169921875e-05 ???????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0010488033294677734 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.20920133590698242 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.06594444444444442 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26113208_21-08-2025_08_21_34.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26113208 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26113208 AND mptpi.`type`=3594 To do Qualite : 0.14344428077066965 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26114421_21-08-2025_09_23_10.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26114421 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26114421 AND mptpi.`type`=3594 To do Qualite : 0.13331061385459533 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26114424_21-08-2025_09_11_26.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26114424 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26114424 AND mptpi.`type`=3594 To do Qualite : 0.06449963563100139 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26115111_21-08-2025_10_02_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26115111 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26115111 AND mptpi.`type`=3594 To do Qualite : 0.037599160420340956 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26115112_21-08-2025_10_12_16.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26115112 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26115112 AND mptpi.`type`=3594 To do Qualite : 0.05686577690972222 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26115114_21-08-2025_09_51_52.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26115114 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26115114 AND mptpi.`type`=3594 To do Qualite : 0.09280790004748335 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26116300_21-08-2025_10_41_51.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26116300 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26116300 AND mptpi.`type`=3594 To do Qualite : 0.19969962062757196 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26116302_21-08-2025_10_52_12.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26116302 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26116302 AND mptpi.`type`=3594 To do Qualite : 0.1368191189236111 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26118869_21-08-2025_21_21_25.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26118869 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26118869 AND mptpi.`type`=3594 To do Qualite : 0.07821421682098766 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26118874_21-08-2025_21_11_29.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26118874 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26118874 AND mptpi.`type`=3594 To do Qualite : 0.17981294608410492 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26121460_21-08-2025_22_12_06.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26121460 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26121460 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'21082025': {'nb_upload': 16, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1378209803, 1378209801, 1378209799, 1378209797, 1378209795, 1378209787, 1378209784, 1378209783, 1378209782, 1378209780, 1378209779, 1378209762, 1378209761, 1378209760, 1378209759, 1378209757] Looping around the photos to save general results len do output : 1 /26121460Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209803', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209801', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209799', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209797', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209795', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209787', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209784', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209783', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209782', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209780', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209779', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209762', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209761', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209760', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209759', None, None, None, None, None, '3568988') ('3318', None, None, None, None, None, None, None, '3568988') ('3318', '26121460', '1378209757', None, None, None, None, None, '3568988') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 17 time used for this insertion : 0.01528787612915039 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.0188121795654297 time spend to save output : 0.015540838241577148 total time spend for step 10 : 1.0343530178070068 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 16 set_done_treatment 68.49user 23.57system 1:46.70elapsed 86%CPU (0avgtext+0avgdata 2869508maxresident)k 411672inputs+44456outputs (36major+1857799minor)pagefaults 0swaps