python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3880609 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['2927940'] with mtr_portfolio_ids : ['23219689'] and first list_photo_ids : [] new path : /proc/3880609/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 9 ; length of list_pids : 9 ; length of list_args : 9 time to download the photos : 1.524364948272705 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Wed May 21 19:00:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 3943 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-05-21 19:00:32.489358: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-05-21 19:00:32.515162: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-05-21 19:00:32.517398: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fea44000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-05-21 19:00:32.517461: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-05-21 19:00:32.521656: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-05-21 19:00:32.700229: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x425bfa60 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-05-21 19:00:32.700273: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-05-21 19:00:32.701194: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-21 19:00:32.701552: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-21 19:00:32.703876: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-21 19:00:32.706148: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-21 19:00:32.706565: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-21 19:00:32.709208: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-21 19:00:32.710716: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-21 19:00:32.715927: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-21 19:00:32.717422: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-21 19:00:32.717523: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-21 19:00:32.718289: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-21 19:00:32.718310: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-21 19:00:32.718324: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-21 19:00:32.719642: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3491 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-05-21 19:00:33.037964: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-21 19:00:33.038056: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-21 19:00:33.038080: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-21 19:00:33.038102: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-21 19:00:33.038123: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-21 19:00:33.038144: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-21 19:00:33.038165: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-21 19:00:33.038186: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-21 19:00:33.039273: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-21 19:00:33.040315: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-21 19:00:33.040350: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-21 19:00:33.040370: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-21 19:00:33.040388: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-21 19:00:33.040405: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-21 19:00:33.040423: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-21 19:00:33.040441: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-21 19:00:33.040459: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-21 19:00:33.041513: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-21 19:00:33.041545: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-21 19:00:33.041556: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-21 19:00:33.041565: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-21 19:00:33.042677: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3491 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-05-21 19:00:44.770056: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-21 19:00:44.966368: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-21 19:00:46.596318: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.91G (3125673984 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 19:00:47.322143: W tensorflow/core/common_runtime/bfc_allocator.cc:311] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you'd like to disable this feature. 2025-05-21 19:00:47.386553: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.91G (3125673984 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 19:00:47.386613: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.67GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-21 19:00:47.387743: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.91G (3125673984 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 19:00:47.387775: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.67GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-21 19:00:47.394610: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.29GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-21 19:00:47.394638: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.29GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-21 19:00:47.400926: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.91G (3125673984 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 19:00:47.400946: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 1.78GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-21 19:00:47.401577: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.91G (3125673984 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 19:00:47.401593: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 1.78GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-21 19:00:47.451964: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.91G (3125673984 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 19:00:47.452029: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.26GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-21 19:00:47.452663: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.91G (3125673984 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 19:00:47.452678: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.26GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-21 19:00:47.460631: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.91G (3125673984 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 19:00:47.460654: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 417.62MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-21 19:00:47.461241: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.91G (3125673984 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 19:00:47.461256: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 417.62MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-21 19:00:47.476920: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.91G (3125673984 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 19:00:47.477729: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.91G (3125673984 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 19:00:47.478523: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.91G (3125673984 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 19:00:47.479326: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.91G (3125673984 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 19:00:47.483712: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.91G (3125673984 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 19:00:47.484359: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.91G (3125673984 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 19:00:47.484973: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.91G (3125673984 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 19:00:47.485553: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.91G (3125673984 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 19:00:47.486497: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.91G (3125673984 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 19:00:47.486513: W tensorflow/core/kernels/gpu_utils.cc:49] Failed to allocate memory for convolution redzone checking; skipping this check. This is benign and only means that we won't check cudnn for out-of-bounds reads and writes. This message will only be printed once. 2025-05-21 19:00:47.496863: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.91G (3125673984 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 19:00:47.497470: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.91G (3125673984 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 19:00:47.506451: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.91G (3125673984 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 19:00:47.507051: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.91G (3125673984 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 19:00:47.507646: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.91G (3125673984 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 19:00:47.508229: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.91G (3125673984 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 19:00:47.508820: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.91G (3125673984 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 19:00:47.509404: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.91G (3125673984 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 9 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 38 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 3.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 46 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 13.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 30 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 27 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 25.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 12.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 24 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 9.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 49 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 26.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 37 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 30 Detection mask done ! Trying to reset tf kernel 3882692 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 2973 tf kernel not reseted sub process len(results) : 9 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 9 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 4166 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0007994174957275391 nb_pixel_total : 16816 time to create 1 rle with old method : 0.01837301254272461 length of segment : 173 time for calcul the mask position with numpy : 0.0004875659942626953 nb_pixel_total : 19993 time to create 1 rle with old method : 0.021552324295043945 length of segment : 189 time for calcul the mask position with numpy : 0.0005087852478027344 nb_pixel_total : 19208 time to create 1 rle with old method : 0.02116703987121582 length of segment : 201 time for calcul the mask position with numpy : 0.0010559558868408203 nb_pixel_total : 49105 time to create 1 rle with old method : 0.05245542526245117 length of segment : 287 time for calcul the mask position with numpy : 0.00077056884765625 nb_pixel_total : 45575 time to create 1 rle with old method : 0.04884696006774902 length of segment : 168 time for calcul the mask position with numpy : 0.0010249614715576172 nb_pixel_total : 41149 time to create 1 rle with old method : 0.044060468673706055 length of segment : 342 time for calcul the mask position with numpy : 0.00031447410583496094 nb_pixel_total : 15214 time to create 1 rle with old method : 0.016312122344970703 length of segment : 130 time for calcul the mask position with numpy : 0.0016179084777832031 nb_pixel_total : 71298 time to create 1 rle with old method : 0.07525753974914551 length of segment : 372 time for calcul the mask position with numpy : 0.0006918907165527344 nb_pixel_total : 42281 time to create 1 rle with old method : 0.04406595230102539 length of segment : 208 time for calcul the mask position with numpy : 0.0044939517974853516 nb_pixel_total : 167689 time to create 1 rle with new method : 0.012381553649902344 length of segment : 550 time for calcul the mask position with numpy : 0.0011887550354003906 nb_pixel_total : 66498 time to create 1 rle with old method : 0.0677645206451416 length of segment : 320 time for calcul the mask position with numpy : 0.0008666515350341797 nb_pixel_total : 37593 time to create 1 rle with old method : 0.03983283042907715 length of segment : 287 time for calcul the mask position with numpy : 0.0011522769927978516 nb_pixel_total : 47169 time to create 1 rle with old method : 0.059459686279296875 length of segment : 219 time for calcul the mask position with numpy : 0.0007956027984619141 nb_pixel_total : 32123 time to create 1 rle with old method : 0.037148237228393555 length of segment : 193 time for calcul the mask position with numpy : 0.0002446174621582031 nb_pixel_total : 10320 time to create 1 rle with old method : 0.011722087860107422 length of segment : 134 time for calcul the mask position with numpy : 0.0013895034790039062 nb_pixel_total : 75329 time to create 1 rle with old method : 0.0876169204711914 length of segment : 299 time for calcul the mask position with numpy : 0.0008003711700439453 nb_pixel_total : 35855 time to create 1 rle with old method : 0.04013776779174805 length of segment : 180 time for calcul the mask position with numpy : 0.0009484291076660156 nb_pixel_total : 44918 time to create 1 rle with old method : 0.05147504806518555 length of segment : 287 time for calcul the mask position with numpy : 0.0004949569702148438 nb_pixel_total : 32754 time to create 1 rle with old method : 0.040787458419799805 length of segment : 147 time for calcul the mask position with numpy : 0.0003075599670410156 nb_pixel_total : 12216 time to create 1 rle with old method : 0.01419210433959961 length of segment : 96 time for calcul the mask position with numpy : 0.001234292984008789 nb_pixel_total : 59788 time to create 1 rle with old method : 0.0703420639038086 length of segment : 307 time for calcul the mask position with numpy : 0.0011415481567382812 nb_pixel_total : 56313 time to create 1 rle with old method : 0.06985640525817871 length of segment : 229 time for calcul the mask position with numpy : 0.0004487037658691406 nb_pixel_total : 22522 time to create 1 rle with old method : 0.02522754669189453 length of segment : 134 time for calcul the mask position with numpy : 0.0008616447448730469 nb_pixel_total : 35499 time to create 1 rle with old method : 0.03938126564025879 length of segment : 355 time for calcul the mask position with numpy : 0.00020170211791992188 nb_pixel_total : 7110 time to create 1 rle with old method : 0.007994651794433594 length of segment : 111 time for calcul the mask position with numpy : 0.000865936279296875 nb_pixel_total : 52419 time to create 1 rle with old method : 0.06044745445251465 length of segment : 331 time for calcul the mask position with numpy : 0.0018439292907714844 nb_pixel_total : 22530 time to create 1 rle with old method : 0.028191089630126953 length of segment : 378 time for calcul the mask position with numpy : 0.013321399688720703 nb_pixel_total : 233410 time to create 1 rle with new method : 0.027582645416259766 length of segment : 668 time for calcul the mask position with numpy : 0.0008323192596435547 nb_pixel_total : 13860 time to create 1 rle with old method : 0.015187978744506836 length of segment : 262 time for calcul the mask position with numpy : 0.002633333206176758 nb_pixel_total : 57141 time to create 1 rle with old method : 0.06374621391296387 length of segment : 295 time for calcul the mask position with numpy : 0.0014307498931884766 nb_pixel_total : 17443 time to create 1 rle with old method : 0.02006387710571289 length of segment : 197 time for calcul the mask position with numpy : 0.0010788440704345703 nb_pixel_total : 22479 time to create 1 rle with old method : 0.025332927703857422 length of segment : 205 time for calcul the mask position with numpy : 0.0025103092193603516 nb_pixel_total : 52971 time to create 1 rle with old method : 0.060207366943359375 length of segment : 268 time for calcul the mask position with numpy : 0.001199960708618164 nb_pixel_total : 23836 time to create 1 rle with old method : 0.027258634567260742 length of segment : 257 time for calcul the mask position with numpy : 0.0022592544555664062 nb_pixel_total : 61057 time to create 1 rle with old method : 0.06700873374938965 length of segment : 238 time for calcul the mask position with numpy : 0.0005879402160644531 nb_pixel_total : 11575 time to create 1 rle with old method : 0.013000249862670898 length of segment : 148 time for calcul the mask position with numpy : 0.0005373954772949219 nb_pixel_total : 14043 time to create 1 rle with old method : 0.015084505081176758 length of segment : 216 time for calcul the mask position with numpy : 0.002247333526611328 nb_pixel_total : 74151 time to create 1 rle with old method : 0.07866811752319336 length of segment : 473 time for calcul the mask position with numpy : 0.0014595985412597656 nb_pixel_total : 38935 time to create 1 rle with old method : 0.04404568672180176 length of segment : 232 time for calcul the mask position with numpy : 0.005549192428588867 nb_pixel_total : 194340 time to create 1 rle with new method : 0.009477376937866211 length of segment : 536 time for calcul the mask position with numpy : 0.0016148090362548828 nb_pixel_total : 32541 time to create 1 rle with old method : 0.03647112846374512 length of segment : 192 time for calcul the mask position with numpy : 0.0008695125579833984 nb_pixel_total : 22834 time to create 1 rle with old method : 0.026142120361328125 length of segment : 174 time for calcul the mask position with numpy : 0.0006885528564453125 nb_pixel_total : 12639 time to create 1 rle with old method : 0.014431476593017578 length of segment : 194 time for calcul the mask position with numpy : 0.0013952255249023438 nb_pixel_total : 29483 time to create 1 rle with old method : 0.033879995346069336 length of segment : 272 time for calcul the mask position with numpy : 0.001216888427734375 nb_pixel_total : 45856 time to create 1 rle with old method : 0.05157732963562012 length of segment : 162 time for calcul the mask position with numpy : 0.0005750656127929688 nb_pixel_total : 16381 time to create 1 rle with old method : 0.019341707229614258 length of segment : 153 time for calcul the mask position with numpy : 0.018390893936157227 nb_pixel_total : 322457 time to create 1 rle with new method : 0.2901797294616699 length of segment : 897 time for calcul the mask position with numpy : 0.0020253658294677734 nb_pixel_total : 68729 time to create 1 rle with old method : 0.07199311256408691 length of segment : 427 time for calcul the mask position with numpy : 0.004415988922119141 nb_pixel_total : 99850 time to create 1 rle with old method : 0.11453962326049805 length of segment : 601 time for calcul the mask position with numpy : 0.0012538433074951172 nb_pixel_total : 37430 time to create 1 rle with old method : 0.04109954833984375 length of segment : 274 time for calcul the mask position with numpy : 0.0017042160034179688 nb_pixel_total : 46173 time to create 1 rle with old method : 0.05103635787963867 length of segment : 473 time for calcul the mask position with numpy : 0.0008697509765625 nb_pixel_total : 22876 time to create 1 rle with old method : 0.025182485580444336 length of segment : 236 time for calcul the mask position with numpy : 0.0011034011840820312 nb_pixel_total : 32267 time to create 1 rle with old method : 0.035806894302368164 length of segment : 203 time for calcul the mask position with numpy : 0.0009374618530273438 nb_pixel_total : 31538 time to create 1 rle with old method : 0.0350184440612793 length of segment : 217 time for calcul the mask position with numpy : 0.0007016658782958984 nb_pixel_total : 22541 time to create 1 rle with old method : 0.025220632553100586 length of segment : 159 time for calcul the mask position with numpy : 0.0010540485382080078 nb_pixel_total : 28392 time to create 1 rle with old method : 0.03318333625793457 length of segment : 204 time for calcul the mask position with numpy : 0.005583047866821289 nb_pixel_total : 206578 time to create 1 rle with new method : 0.008935689926147461 length of segment : 507 time for calcul the mask position with numpy : 0.0010385513305664062 nb_pixel_total : 21256 time to create 1 rle with old method : 0.023917436599731445 length of segment : 218 time for calcul the mask position with numpy : 0.0007779598236083984 nb_pixel_total : 19876 time to create 1 rle with old method : 0.022237300872802734 length of segment : 227 time for calcul the mask position with numpy : 0.006284475326538086 nb_pixel_total : 237448 time to create 1 rle with new method : 0.011535882949829102 length of segment : 739 time for calcul the mask position with numpy : 0.0008716583251953125 nb_pixel_total : 23579 time to create 1 rle with old method : 0.025614261627197266 length of segment : 186 time for calcul the mask position with numpy : 0.0008952617645263672 nb_pixel_total : 26182 time to create 1 rle with old method : 0.029102563858032227 length of segment : 263 time for calcul the mask position with numpy : 0.0034105777740478516 nb_pixel_total : 83544 time to create 1 rle with old method : 0.08864021301269531 length of segment : 622 time for calcul the mask position with numpy : 0.0031104087829589844 nb_pixel_total : 148774 time to create 1 rle with old method : 0.165968656539917 length of segment : 397 time for calcul the mask position with numpy : 0.0015721321105957031 nb_pixel_total : 53101 time to create 1 rle with old method : 0.05991959571838379 length of segment : 351 time for calcul the mask position with numpy : 0.0009565353393554688 nb_pixel_total : 23805 time to create 1 rle with old method : 0.026787519454956055 length of segment : 202 time for calcul the mask position with numpy : 0.00226593017578125 nb_pixel_total : 90597 time to create 1 rle with old method : 0.09901857376098633 length of segment : 405 time for calcul the mask position with numpy : 0.0008778572082519531 nb_pixel_total : 45167 time to create 1 rle with old method : 0.05136704444885254 length of segment : 178 time for calcul the mask position with numpy : 0.0012736320495605469 nb_pixel_total : 56802 time to create 1 rle with old method : 0.0625603199005127 length of segment : 332 time for calcul the mask position with numpy : 0.004413127899169922 nb_pixel_total : 275284 time to create 1 rle with new method : 0.010161638259887695 length of segment : 448 time for calcul the mask position with numpy : 0.003645181655883789 nb_pixel_total : 214474 time to create 1 rle with new method : 0.010045528411865234 length of segment : 549 time for calcul the mask position with numpy : 0.0008790493011474609 nb_pixel_total : 21767 time to create 1 rle with old method : 0.023816823959350586 length of segment : 186 time for calcul the mask position with numpy : 0.002619028091430664 nb_pixel_total : 101313 time to create 1 rle with old method : 0.10992741584777832 length of segment : 364 time for calcul the mask position with numpy : 0.011182546615600586 nb_pixel_total : 366338 time to create 1 rle with new method : 0.03125429153442383 length of segment : 960 time for calcul the mask position with numpy : 0.007396697998046875 nb_pixel_total : 217088 time to create 1 rle with new method : 0.0101165771484375 length of segment : 522 time for calcul the mask position with numpy : 0.001087188720703125 nb_pixel_total : 29014 time to create 1 rle with old method : 0.03331351280212402 length of segment : 202 time for calcul the mask position with numpy : 0.00439143180847168 nb_pixel_total : 131848 time to create 1 rle with old method : 0.1439528465270996 length of segment : 519 time for calcul the mask position with numpy : 0.0005922317504882812 nb_pixel_total : 24181 time to create 1 rle with old method : 0.025951623916625977 length of segment : 136 time for calcul the mask position with numpy : 0.0011830329895019531 nb_pixel_total : 51172 time to create 1 rle with old method : 0.05598258972167969 length of segment : 247 time for calcul the mask position with numpy : 0.0020105838775634766 nb_pixel_total : 57421 time to create 1 rle with old method : 0.06197023391723633 length of segment : 579 time for calcul the mask position with numpy : 0.0002987384796142578 nb_pixel_total : 15357 time to create 1 rle with old method : 0.016263484954833984 length of segment : 123 time for calcul the mask position with numpy : 0.001009225845336914 nb_pixel_total : 21257 time to create 1 rle with old method : 0.024105072021484375 length of segment : 167 time for calcul the mask position with numpy : 0.00047850608825683594 nb_pixel_total : 5764 time to create 1 rle with old method : 0.007011890411376953 length of segment : 91 time for calcul the mask position with numpy : 0.0017077922821044922 nb_pixel_total : 31253 time to create 1 rle with old method : 0.035405874252319336 length of segment : 235 time for calcul the mask position with numpy : 0.0019032955169677734 nb_pixel_total : 40843 time to create 1 rle with old method : 0.04409289360046387 length of segment : 278 time for calcul the mask position with numpy : 0.0037202835083007812 nb_pixel_total : 49762 time to create 1 rle with old method : 0.05432629585266113 length of segment : 258 time for calcul the mask position with numpy : 0.0003147125244140625 nb_pixel_total : 5831 time to create 1 rle with old method : 0.006496906280517578 length of segment : 69 time for calcul the mask position with numpy : 0.0012021064758300781 nb_pixel_total : 19606 time to create 1 rle with old method : 0.021698713302612305 length of segment : 150 time for calcul the mask position with numpy : 0.0011761188507080078 nb_pixel_total : 16015 time to create 1 rle with old method : 0.01741194725036621 length of segment : 233 time for calcul the mask position with numpy : 0.001558542251586914 nb_pixel_total : 38165 time to create 1 rle with old method : 0.04317045211791992 length of segment : 184 time for calcul the mask position with numpy : 0.0009784698486328125 nb_pixel_total : 17405 time to create 1 rle with old method : 0.019276142120361328 length of segment : 333 time for calcul the mask position with numpy : 0.0005054473876953125 nb_pixel_total : 14058 time to create 1 rle with old method : 0.015825748443603516 length of segment : 122 time for calcul the mask position with numpy : 0.0010640621185302734 nb_pixel_total : 24387 time to create 1 rle with old method : 0.027105331420898438 length of segment : 171 time for calcul the mask position with numpy : 0.0005927085876464844 nb_pixel_total : 11164 time to create 1 rle with old method : 0.012393474578857422 length of segment : 248 time for calcul the mask position with numpy : 0.0007534027099609375 nb_pixel_total : 14657 time to create 1 rle with old method : 0.01645183563232422 length of segment : 119 time for calcul the mask position with numpy : 0.0011982917785644531 nb_pixel_total : 27698 time to create 1 rle with old method : 0.030202865600585938 length of segment : 177 time for calcul the mask position with numpy : 0.0008044242858886719 nb_pixel_total : 19976 time to create 1 rle with old method : 0.02140021324157715 length of segment : 165 time for calcul the mask position with numpy : 0.00041103363037109375 nb_pixel_total : 5228 time to create 1 rle with old method : 0.0057904720306396484 length of segment : 147 time for calcul the mask position with numpy : 0.0005834102630615234 nb_pixel_total : 13755 time to create 1 rle with old method : 0.015533208847045898 length of segment : 156 time for calcul the mask position with numpy : 0.0017154216766357422 nb_pixel_total : 55232 time to create 1 rle with old method : 0.06053948402404785 length of segment : 270 time for calcul the mask position with numpy : 0.0009469985961914062 nb_pixel_total : 19500 time to create 1 rle with old method : 0.03367304801940918 length of segment : 138 time for calcul the mask position with numpy : 0.0005471706390380859 nb_pixel_total : 5209 time to create 1 rle with old method : 0.008076190948486328 length of segment : 122 time for calcul the mask position with numpy : 0.0018978118896484375 nb_pixel_total : 41741 time to create 1 rle with old method : 0.04741859436035156 length of segment : 187 time for calcul the mask position with numpy : 0.00033783912658691406 nb_pixel_total : 6528 time to create 1 rle with old method : 0.007224321365356445 length of segment : 119 time for calcul the mask position with numpy : 0.0008699893951416016 nb_pixel_total : 17608 time to create 1 rle with old method : 0.02006673812866211 length of segment : 165 time for calcul the mask position with numpy : 0.0014166831970214844 nb_pixel_total : 34194 time to create 1 rle with old method : 0.03751063346862793 length of segment : 264 time for calcul the mask position with numpy : 0.0008790493011474609 nb_pixel_total : 20900 time to create 1 rle with old method : 0.023416519165039062 length of segment : 258 time for calcul the mask position with numpy : 0.0021915435791015625 nb_pixel_total : 51193 time to create 1 rle with old method : 0.05574989318847656 length of segment : 385 time for calcul the mask position with numpy : 0.0024657249450683594 nb_pixel_total : 61557 time to create 1 rle with old method : 0.06795597076416016 length of segment : 200 time for calcul the mask position with numpy : 0.002589702606201172 nb_pixel_total : 66619 time to create 1 rle with old method : 0.07303214073181152 length of segment : 296 time for calcul the mask position with numpy : 0.008995294570922852 nb_pixel_total : 227317 time to create 1 rle with new method : 0.013962984085083008 length of segment : 726 time for calcul the mask position with numpy : 0.0009469985961914062 nb_pixel_total : 17820 time to create 1 rle with old method : 0.019550561904907227 length of segment : 205 time for calcul the mask position with numpy : 0.0004458427429199219 nb_pixel_total : 8190 time to create 1 rle with old method : 0.009261369705200195 length of segment : 151 time for calcul the mask position with numpy : 0.0013790130615234375 nb_pixel_total : 32181 time to create 1 rle with old method : 0.035604000091552734 length of segment : 186 time for calcul the mask position with numpy : 0.0008463859558105469 nb_pixel_total : 12029 time to create 1 rle with old method : 0.013770818710327148 length of segment : 183 time for calcul the mask position with numpy : 0.0005085468292236328 nb_pixel_total : 9978 time to create 1 rle with old method : 0.011559724807739258 length of segment : 97 time for calcul the mask position with numpy : 0.0009338855743408203 nb_pixel_total : 25547 time to create 1 rle with old method : 0.028334617614746094 length of segment : 203 time for calcul the mask position with numpy : 0.002599954605102539 nb_pixel_total : 72363 time to create 1 rle with old method : 0.08054065704345703 length of segment : 406 time for calcul the mask position with numpy : 0.0004086494445800781 nb_pixel_total : 11782 time to create 1 rle with old method : 0.013936996459960938 length of segment : 116 time for calcul the mask position with numpy : 0.0004954338073730469 nb_pixel_total : 10765 time to create 1 rle with old method : 0.012400627136230469 length of segment : 128 time for calcul the mask position with numpy : 0.0010194778442382812 nb_pixel_total : 21884 time to create 1 rle with old method : 0.024853229522705078 length of segment : 228 time for calcul the mask position with numpy : 0.001123189926147461 nb_pixel_total : 25520 time to create 1 rle with old method : 0.02939319610595703 length of segment : 203 time for calcul the mask position with numpy : 0.0014424324035644531 nb_pixel_total : 21299 time to create 1 rle with old method : 0.025954008102416992 length of segment : 274 time for calcul the mask position with numpy : 0.0014140605926513672 nb_pixel_total : 34439 time to create 1 rle with old method : 0.038177490234375 length of segment : 171 time for calcul the mask position with numpy : 0.0008180141448974609 nb_pixel_total : 22107 time to create 1 rle with old method : 0.024676799774169922 length of segment : 195 time for calcul the mask position with numpy : 0.0018393993377685547 nb_pixel_total : 55879 time to create 1 rle with old method : 0.061078786849975586 length of segment : 256 time for calcul the mask position with numpy : 0.0015654563903808594 nb_pixel_total : 15645 time to create 1 rle with old method : 0.018303632736206055 length of segment : 384 time for calcul the mask position with numpy : 0.0012104511260986328 nb_pixel_total : 22087 time to create 1 rle with old method : 0.025056123733520508 length of segment : 279 time for calcul the mask position with numpy : 0.0002601146697998047 nb_pixel_total : 10136 time to create 1 rle with old method : 0.011646747589111328 length of segment : 114 time for calcul the mask position with numpy : 0.00039005279541015625 nb_pixel_total : 13415 time to create 1 rle with old method : 0.015061616897583008 length of segment : 147 time for calcul the mask position with numpy : 0.0008854866027832031 nb_pixel_total : 25358 time to create 1 rle with old method : 0.029530763626098633 length of segment : 166 time for calcul the mask position with numpy : 0.0006451606750488281 nb_pixel_total : 14375 time to create 1 rle with old method : 0.01619100570678711 length of segment : 143 time for calcul the mask position with numpy : 0.0012116432189941406 nb_pixel_total : 31162 time to create 1 rle with old method : 0.035298824310302734 length of segment : 209 time for calcul the mask position with numpy : 0.001016855239868164 nb_pixel_total : 22758 time to create 1 rle with old method : 0.02534961700439453 length of segment : 192 time for calcul the mask position with numpy : 0.00047850608825683594 nb_pixel_total : 13740 time to create 1 rle with old method : 0.015353202819824219 length of segment : 120 time for calcul the mask position with numpy : 0.0004131793975830078 nb_pixel_total : 8591 time to create 1 rle with old method : 0.010053873062133789 length of segment : 115 time for calcul the mask position with numpy : 0.0021369457244873047 nb_pixel_total : 83959 time to create 1 rle with old method : 0.09223556518554688 length of segment : 149 time for calcul the mask position with numpy : 0.0075054168701171875 nb_pixel_total : 244941 time to create 1 rle with new method : 0.013952255249023438 length of segment : 629 time for calcul the mask position with numpy : 0.0005033016204833984 nb_pixel_total : 9474 time to create 1 rle with old method : 0.010809659957885742 length of segment : 164 time for calcul the mask position with numpy : 0.0008330345153808594 nb_pixel_total : 31423 time to create 1 rle with old method : 0.03546881675720215 length of segment : 194 time for calcul the mask position with numpy : 0.00624537467956543 nb_pixel_total : 194159 time to create 1 rle with new method : 0.012634992599487305 length of segment : 639 time for calcul the mask position with numpy : 0.0011010169982910156 nb_pixel_total : 45543 time to create 1 rle with old method : 0.0509793758392334 length of segment : 260 time for calcul the mask position with numpy : 0.006613492965698242 nb_pixel_total : 182333 time to create 1 rle with new method : 0.014568090438842773 length of segment : 876 time for calcul the mask position with numpy : 0.002530813217163086 nb_pixel_total : 77258 time to create 1 rle with old method : 0.08463883399963379 length of segment : 379 time for calcul the mask position with numpy : 0.0006165504455566406 nb_pixel_total : 13277 time to create 1 rle with old method : 0.015194416046142578 length of segment : 150 time for calcul the mask position with numpy : 0.0010695457458496094 nb_pixel_total : 30925 time to create 1 rle with old method : 0.03380179405212402 length of segment : 293 time for calcul the mask position with numpy : 0.0035033226013183594 nb_pixel_total : 93083 time to create 1 rle with old method : 0.1280684471130371 length of segment : 481 time spent for convertir_results : 12.00727915763855 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 147 chid ids of type : 3594 Number RLEs to save : 40440 save missing photos in datou_result : time spend for datou_step_exec : 76.05472183227539 time spend to save output : 2.6261119842529297 total time spend for step 1 : 78.68083381652832 step2:crop_condition Wed May 21 19:01:48 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 9 ! batch 1 Loaded 147 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 89 About to insert : list_path_to_insert length 89 new photo from crops ! About to upload 89 photos upload in portfolio : 3736932 init cache_photo without model_param we have 89 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747846928_3880609 we have uploaded 89 photos in the portfolio 3736932 time of upload the photos Elapsed time : 23.923487424850464 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 21 About to insert : list_path_to_insert length 21 new photo from crops ! About to upload 21 photos upload in portfolio : 3736932 init cache_photo without model_param we have 21 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747846958_3880609 we have uploaded 21 photos in the portfolio 3736932 time of upload the photos Elapsed time : 4.619283437728882 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747846964_3880609 we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.5660889148712158 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 36 About to insert : list_path_to_insert length 36 new photo from crops ! About to upload 36 photos upload in portfolio : 3736932 init cache_photo without model_param we have 36 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747846978_3880609 we have uploaded 36 photos in the portfolio 3736932 time of upload the photos Elapsed time : 8.787827968597412 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1359972916, 1359972904, 1359972890, 1359972869, 1359972863, 1359972811, 1359972808, 1359972805, 1359972683] Looping around the photos to save general results len do output : 147 /1360114105Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114107Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114108Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114109Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114111Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114112Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114113Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114115Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114116Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114117Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114119Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114120Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114121Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114123Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114124Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114125Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114127Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114128Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114129Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114132Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114133Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114134Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114136Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114137Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114138Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114140Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114141Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114142Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114144Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114145Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114146Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114147Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114149Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114150Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114151Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114153Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114155Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114156Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114158Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114160Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114161Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114163Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114165Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114166Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114168Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114170Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114171Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114173Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114175Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114176Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114178Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114180Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114181Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114184Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114185Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114187Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114189Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114190Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114192Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114194Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114195Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114197Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114199Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114200Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114202Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114204Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114205Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114207Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114209Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114210Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114212Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114214Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114215Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114217Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114219Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114220Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114222Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114224Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114225Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114227Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114229Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114230Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114231Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114232Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114233Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114234Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114235Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114236Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114237Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114240Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114241Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114242Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114243Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114244Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114245Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114246Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114247Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114248Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114249Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114250Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114251Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114252Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114253Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114254Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114255Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114256Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114257Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114258Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114259Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114260Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114271Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114516Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114518Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114520Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114521Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114523Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114525Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114526Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114528Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114530Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114531Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114534Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114536Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114537Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114539Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114541Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114542Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114544Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114546Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114547Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114549Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114551Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114552Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114554Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114556Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114557Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114560Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114562Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114563Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114565Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114567Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114568Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114570Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114572Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114573Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114575Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360114577Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972916', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972904', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972890', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972869', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972863', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972811', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972808', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972805', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972683', None, None, None, None, None, '2927940') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 450 time used for this insertion : 0.034961700439453125 save_final save missing photos in datou_result : time spend for datou_step_exec : 78.76851749420166 time spend to save output : 0.038231849670410156 total time spend for step 2 : 78.80674934387207 step3:rle_unique_nms_with_priority Wed May 21 19:03:07 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 147 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 20 nb_hashtags : 3 time to prepare the origin masks : 13.245500087738037 time for calcul the mask position with numpy : 1.0159845352172852 nb_pixel_total : 6203852 time to create 1 rle with new method : 1.3674170970916748 time for calcul the mask position with numpy : 0.032768964767456055 nb_pixel_total : 12216 time to create 1 rle with old method : 0.013555049896240234 time for calcul the mask position with numpy : 0.02146768569946289 nb_pixel_total : 32754 time to create 1 rle with old method : 0.03588509559631348 time for calcul the mask position with numpy : 0.021029949188232422 nb_pixel_total : 8784 time to create 1 rle with old method : 0.009404897689819336 time for calcul the mask position with numpy : 0.020966768264770508 nb_pixel_total : 35855 time to create 1 rle with old method : 0.0388946533203125 time for calcul the mask position with numpy : 0.021971702575683594 nb_pixel_total : 74748 time to create 1 rle with old method : 0.08034133911132812 time for calcul the mask position with numpy : 0.02014636993408203 nb_pixel_total : 10320 time to create 1 rle with old method : 0.010936498641967773 time for calcul the mask position with numpy : 0.020924806594848633 nb_pixel_total : 32123 time to create 1 rle with old method : 0.03550219535827637 time for calcul the mask position with numpy : 0.02154707908630371 nb_pixel_total : 47169 time to create 1 rle with old method : 0.05039787292480469 time for calcul the mask position with numpy : 0.020473957061767578 nb_pixel_total : 37593 time to create 1 rle with old method : 0.04031825065612793 time for calcul the mask position with numpy : 0.020884275436401367 nb_pixel_total : 66498 time to create 1 rle with old method : 0.07244467735290527 time for calcul the mask position with numpy : 0.022696495056152344 nb_pixel_total : 167689 time to create 1 rle with new method : 0.8267600536346436 time for calcul the mask position with numpy : 0.02044963836669922 nb_pixel_total : 42281 time to create 1 rle with old method : 0.04512834548950195 time for calcul the mask position with numpy : 0.02201056480407715 nb_pixel_total : 71298 time to create 1 rle with old method : 0.08066153526306152 time for calcul the mask position with numpy : 0.021682262420654297 nb_pixel_total : 15214 time to create 1 rle with old method : 0.016349077224731445 time for calcul the mask position with numpy : 0.02085256576538086 nb_pixel_total : 41149 time to create 1 rle with old method : 0.043671369552612305 time for calcul the mask position with numpy : 0.020472049713134766 nb_pixel_total : 45575 time to create 1 rle with old method : 0.04789471626281738 time for calcul the mask position with numpy : 0.02077960968017578 nb_pixel_total : 49105 time to create 1 rle with old method : 0.050904035568237305 time for calcul the mask position with numpy : 0.020637989044189453 nb_pixel_total : 19208 time to create 1 rle with old method : 0.020762205123901367 time for calcul the mask position with numpy : 0.020629167556762695 nb_pixel_total : 19993 time to create 1 rle with old method : 0.022644758224487305 time for calcul the mask position with numpy : 0.020971298217773438 nb_pixel_total : 16816 time to create 1 rle with old method : 0.01788949966430664 create new chi : 4.428833246231079 time to delete rle : 0.02585768699645996 batch 1 Loaded 41 chid ids of type : 3594 +++++++++++++++++++++++++++Number RLEs to save : 11394 TO DO : save crop sub photo not yet done ! save time : 0.7775800228118896 nb_obj : 18 nb_hashtags : 3 time to prepare the origin masks : 11.12052059173584 time for calcul the mask position with numpy : 1.6994802951812744 nb_pixel_total : 6219511 time to create 1 rle with new method : 1.0953903198242188 time for calcul the mask position with numpy : 0.020369291305541992 nb_pixel_total : 66733 time to create 1 rle with old method : 0.07002377510070801 time for calcul the mask position with numpy : 0.02044844627380371 nb_pixel_total : 14043 time to create 1 rle with old method : 0.015319347381591797 time for calcul the mask position with numpy : 0.01972365379333496 nb_pixel_total : 11575 time to create 1 rle with old method : 0.012396097183227539 time for calcul the mask position with numpy : 0.02054619789123535 nb_pixel_total : 61057 time to create 1 rle with old method : 0.06435418128967285 time for calcul the mask position with numpy : 0.021647214889526367 nb_pixel_total : 23836 time to create 1 rle with old method : 0.025915145874023438 time for calcul the mask position with numpy : 0.021921634674072266 nb_pixel_total : 52971 time to create 1 rle with old method : 0.057761430740356445 time for calcul the mask position with numpy : 0.02121877670288086 nb_pixel_total : 22479 time to create 1 rle with old method : 0.02471160888671875 time for calcul the mask position with numpy : 0.021175622940063477 nb_pixel_total : 17443 time to create 1 rle with old method : 0.019024133682250977 time for calcul the mask position with numpy : 0.021615266799926758 nb_pixel_total : 57141 time to create 1 rle with old method : 0.06259274482727051 time for calcul the mask position with numpy : 0.021419048309326172 nb_pixel_total : 13860 time to create 1 rle with old method : 0.015353679656982422 time for calcul the mask position with numpy : 0.024224281311035156 nb_pixel_total : 233410 time to create 1 rle with new method : 1.3305237293243408 time for calcul the mask position with numpy : 0.020531892776489258 nb_pixel_total : 22530 time to create 1 rle with old method : 0.02450251579284668 time for calcul the mask position with numpy : 0.02096247673034668 nb_pixel_total : 52419 time to create 1 rle with old method : 0.05539584159851074 time for calcul the mask position with numpy : 0.02012944221496582 nb_pixel_total : 7110 time to create 1 rle with old method : 0.007594108581542969 time for calcul the mask position with numpy : 0.021548032760620117 nb_pixel_total : 35499 time to create 1 rle with old method : 0.03967547416687012 time for calcul the mask position with numpy : 0.02143692970275879 nb_pixel_total : 22522 time to create 1 rle with old method : 0.024736404418945312 time for calcul the mask position with numpy : 0.02144646644592285 nb_pixel_total : 56313 time to create 1 rle with old method : 0.061692237854003906 time for calcul the mask position with numpy : 0.023381948471069336 nb_pixel_total : 59788 time to create 1 rle with old method : 0.06508851051330566 create new chi : 5.213076114654541 time to delete rle : 0.001581430435180664 batch 1 Loaded 37 chid ids of type : 3594 +++++++++++++++++++++++++++Number RLEs to save : 12047 TO DO : save crop sub photo not yet done ! save time : 0.7948613166809082 nb_obj : 14 nb_hashtags : 3 time to prepare the origin masks : 7.864657163619995 time for calcul the mask position with numpy : 1.0227415561676025 nb_pixel_total : 6062669 time to create 1 rle with new method : 1.6232037544250488 time for calcul the mask position with numpy : 0.0198667049407959 nb_pixel_total : 22876 time to create 1 rle with old method : 0.02388167381286621 time for calcul the mask position with numpy : 0.019545555114746094 nb_pixel_total : 46173 time to create 1 rle with old method : 0.0468137264251709 time for calcul the mask position with numpy : 0.020594120025634766 nb_pixel_total : 37430 time to create 1 rle with old method : 0.03774595260620117 time for calcul the mask position with numpy : 0.02070450782775879 nb_pixel_total : 99850 time to create 1 rle with old method : 0.10315084457397461 time for calcul the mask position with numpy : 0.02211284637451172 nb_pixel_total : 68729 time to create 1 rle with old method : 0.07168459892272949 time for calcul the mask position with numpy : 0.022421836853027344 nb_pixel_total : 319504 time to create 1 rle with new method : 1.7002055644989014 time for calcul the mask position with numpy : 0.02008676528930664 nb_pixel_total : 16381 time to create 1 rle with old method : 0.017291784286499023 time for calcul the mask position with numpy : 0.021282434463500977 nb_pixel_total : 45856 time to create 1 rle with old method : 0.05056500434875488 time for calcul the mask position with numpy : 0.021770954132080078 nb_pixel_total : 29483 time to create 1 rle with old method : 0.031639814376831055 time for calcul the mask position with numpy : 0.021007537841796875 nb_pixel_total : 12639 time to create 1 rle with old method : 0.013379096984863281 time for calcul the mask position with numpy : 0.021347522735595703 nb_pixel_total : 22834 time to create 1 rle with old method : 0.024625778198242188 time for calcul the mask position with numpy : 0.02127695083618164 nb_pixel_total : 32541 time to create 1 rle with old method : 0.034945011138916016 time for calcul the mask position with numpy : 0.021554946899414062 nb_pixel_total : 194340 time to create 1 rle with new method : 1.605557918548584 time for calcul the mask position with numpy : 0.020828962326049805 nb_pixel_total : 38935 time to create 1 rle with old method : 0.04062819480895996 create new chi : 6.818727016448975 time to delete rle : 0.0014872550964355469 batch 1 Loaded 29 chid ids of type : 3594 +++++++++++++++++++++++++Number RLEs to save : 11676 TO DO : save crop sub photo not yet done ! save time : 0.7945270538330078 nb_obj : 15 nb_hashtags : 3 time to prepare the origin masks : 10.214786052703857 time for calcul the mask position with numpy : 0.9635231494903564 nb_pixel_total : 6000762 time to create 1 rle with new method : 2.111759662628174 time for calcul the mask position with numpy : 0.03932809829711914 nb_pixel_total : 90597 time to create 1 rle with old method : 0.12269902229309082 time for calcul the mask position with numpy : 0.021252870559692383 nb_pixel_total : 23805 time to create 1 rle with old method : 0.026057720184326172 time for calcul the mask position with numpy : 0.020681381225585938 nb_pixel_total : 53101 time to create 1 rle with old method : 0.057929277420043945 time for calcul the mask position with numpy : 0.02215099334716797 nb_pixel_total : 148774 time to create 1 rle with old method : 0.16112756729125977 time for calcul the mask position with numpy : 0.022064924240112305 nb_pixel_total : 83544 time to create 1 rle with old method : 0.08874154090881348 time for calcul the mask position with numpy : 0.021478891372680664 nb_pixel_total : 26182 time to create 1 rle with old method : 0.027974367141723633 time for calcul the mask position with numpy : 0.020419597625732422 nb_pixel_total : 23579 time to create 1 rle with old method : 0.02484416961669922 time for calcul the mask position with numpy : 0.022938013076782227 nb_pixel_total : 237448 time to create 1 rle with new method : 2.0616614818573 time for calcul the mask position with numpy : 0.020761489868164062 nb_pixel_total : 19876 time to create 1 rle with old method : 0.021101951599121094 time for calcul the mask position with numpy : 0.019548416137695312 nb_pixel_total : 21256 time to create 1 rle with old method : 0.021565914154052734 time for calcul the mask position with numpy : 0.02204132080078125 nb_pixel_total : 206578 time to create 1 rle with new method : 1.2661044597625732 time for calcul the mask position with numpy : 0.02145862579345703 nb_pixel_total : 28392 time to create 1 rle with old method : 0.03170609474182129 time for calcul the mask position with numpy : 0.02111506462097168 nb_pixel_total : 22541 time to create 1 rle with old method : 0.02885723114013672 time for calcul the mask position with numpy : 0.023523569107055664 nb_pixel_total : 31538 time to create 1 rle with old method : 0.04860687255859375 time for calcul the mask position with numpy : 0.022115230560302734 nb_pixel_total : 32267 time to create 1 rle with old method : 0.03574490547180176 create new chi : 7.531327486038208 time to delete rle : 0.0015597343444824219 batch 1 Loaded 31 chid ids of type : 3594 +++++++++++++++Number RLEs to save : 11960 TO DO : save crop sub photo not yet done ! save time : 0.7644314765930176 nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 2.9940309524536133 time for calcul the mask position with numpy : 0.46675729751586914 nb_pixel_total : 6459815 time to create 1 rle with new method : 1.6179735660552979 time for calcul the mask position with numpy : 0.0211484432220459 nb_pixel_total : 213172 time to create 1 rle with new method : 2.385303020477295 time for calcul the mask position with numpy : 0.02276325225830078 nb_pixel_total : 275284 time to create 1 rle with new method : 0.8589329719543457 time for calcul the mask position with numpy : 0.02440047264099121 nb_pixel_total : 56802 time to create 1 rle with old method : 0.07199430465698242 time for calcul the mask position with numpy : 0.022794008255004883 nb_pixel_total : 45167 time to create 1 rle with old method : 0.05069732666015625 create new chi : 5.629448890686035 time to delete rle : 0.0011982917785644531 batch 1 Loaded 9 chid ids of type : 3594 +++++Number RLEs to save : 5069 TO DO : save crop sub photo not yet done ! save time : 0.38773560523986816 nb_obj : 10 nb_hashtags : 3 time to prepare the origin masks : 5.780454158782959 time for calcul the mask position with numpy : 0.6651444435119629 nb_pixel_total : 6036053 time to create 1 rle with new method : 1.583038568496704 time for calcul the mask position with numpy : 0.020240068435668945 nb_pixel_total : 15357 time to create 1 rle with old method : 0.016556262969970703 time for calcul the mask position with numpy : 0.021630525588989258 nb_pixel_total : 56109 time to create 1 rle with old method : 0.06197166442871094 time for calcul the mask position with numpy : 0.022993087768554688 nb_pixel_total : 51172 time to create 1 rle with old method : 0.05643796920776367 time for calcul the mask position with numpy : 0.020967483520507812 nb_pixel_total : 24181 time to create 1 rle with old method : 0.025690555572509766 time for calcul the mask position with numpy : 0.021170377731323242 nb_pixel_total : 131848 time to create 1 rle with old method : 0.14068174362182617 time for calcul the mask position with numpy : 0.02095651626586914 nb_pixel_total : 29014 time to create 1 rle with old method : 0.0311276912689209 time for calcul the mask position with numpy : 0.0220489501953125 nb_pixel_total : 217088 time to create 1 rle with new method : 0.7425906658172607 time for calcul the mask position with numpy : 0.022797346115112305 nb_pixel_total : 366338 time to create 1 rle with new method : 0.842003583908081 time for calcul the mask position with numpy : 0.021570205688476562 nb_pixel_total : 101313 time to create 1 rle with old method : 0.10519671440124512 time for calcul the mask position with numpy : 0.020380258560180664 nb_pixel_total : 21767 time to create 1 rle with old method : 0.02229928970336914 create new chi : 4.588174819946289 time to delete rle : 0.0013468265533447266 batch 1 Loaded 21 chid ids of type : 3594 ++++++++++++Number RLEs to save : 9682 TO DO : save crop sub photo not yet done ! save time : 0.6596477031707764 nb_obj : 35 nb_hashtags : 4 time to prepare the origin masks : 3.5594658851623535 time for calcul the mask position with numpy : 0.6091954708099365 nb_pixel_total : 5991350 time to create 1 rle with new method : 0.8746283054351807 time for calcul the mask position with numpy : 0.026188135147094727 nb_pixel_total : 21257 time to create 1 rle with old method : 0.021819591522216797 time for calcul the mask position with numpy : 0.02643871307373047 nb_pixel_total : 11164 time to create 1 rle with old method : 0.011823177337646484 time for calcul the mask position with numpy : 0.027158737182617188 nb_pixel_total : 14657 time to create 1 rle with old method : 0.014781951904296875 time for calcul the mask position with numpy : 0.026972055435180664 nb_pixel_total : 66619 time to create 1 rle with old method : 0.06856417655944824 time for calcul the mask position with numpy : 0.027677059173583984 nb_pixel_total : 5831 time to create 1 rle with old method : 0.006215810775756836 time for calcul the mask position with numpy : 0.027589082717895508 nb_pixel_total : 12029 time to create 1 rle with old method : 0.012604236602783203 time for calcul the mask position with numpy : 0.026650667190551758 nb_pixel_total : 17405 time to create 1 rle with old method : 0.017904043197631836 time for calcul the mask position with numpy : 0.026509761810302734 nb_pixel_total : 27698 time to create 1 rle with old method : 0.029965639114379883 time for calcul the mask position with numpy : 0.027965784072875977 nb_pixel_total : 17608 time to create 1 rle with old method : 0.018865585327148438 time for calcul the mask position with numpy : 0.0271146297454834 nb_pixel_total : 55232 time to create 1 rle with old method : 0.0551753044128418 time for calcul the mask position with numpy : 0.02619004249572754 nb_pixel_total : 5228 time to create 1 rle with old method : 0.005483865737915039 time for calcul the mask position with numpy : 0.026337623596191406 nb_pixel_total : 19976 time to create 1 rle with old method : 0.02043771743774414 time for calcul the mask position with numpy : 0.02694869041442871 nb_pixel_total : 16130 time to create 1 rle with old method : 0.01677227020263672 time for calcul the mask position with numpy : 0.02648329734802246 nb_pixel_total : 17820 time to create 1 rle with old method : 0.018586397171020508 time for calcul the mask position with numpy : 0.026780128479003906 nb_pixel_total : 16015 time to create 1 rle with old method : 0.016714096069335938 time for calcul the mask position with numpy : 0.026887178421020508 nb_pixel_total : 19500 time to create 1 rle with old method : 0.020392179489135742 time for calcul the mask position with numpy : 0.02735304832458496 nb_pixel_total : 13755 time to create 1 rle with old method : 0.014526605606079102 time for calcul the mask position with numpy : 0.026553630828857422 nb_pixel_total : 31253 time to create 1 rle with old method : 0.03188824653625488 time for calcul the mask position with numpy : 0.02679896354675293 nb_pixel_total : 9978 time to create 1 rle with old method : 0.01105642318725586 time for calcul the mask position with numpy : 0.028264760971069336 nb_pixel_total : 40843 time to create 1 rle with old method : 0.04241800308227539 time for calcul the mask position with numpy : 0.026209115982055664 nb_pixel_total : 41741 time to create 1 rle with old method : 0.04238271713256836 time for calcul the mask position with numpy : 0.026146411895751953 nb_pixel_total : 34194 time to create 1 rle with old method : 0.035172462463378906 time for calcul the mask position with numpy : 0.027469635009765625 nb_pixel_total : 61557 time to create 1 rle with old method : 0.06378889083862305 time for calcul the mask position with numpy : 0.0280458927154541 nb_pixel_total : 19606 time to create 1 rle with old method : 0.02157139778137207 time for calcul the mask position with numpy : 0.02904057502746582 nb_pixel_total : 227317 time to create 1 rle with new method : 0.825690746307373 time for calcul the mask position with numpy : 0.02723860740661621 nb_pixel_total : 8190 time to create 1 rle with old method : 0.008481502532958984 time for calcul the mask position with numpy : 0.02678966522216797 nb_pixel_total : 13098 time to create 1 rle with old method : 0.014009237289428711 time for calcul the mask position with numpy : 0.03164935111999512 nb_pixel_total : 32181 time to create 1 rle with old method : 0.05160808563232422 time for calcul the mask position with numpy : 0.028362751007080078 nb_pixel_total : 38165 time to create 1 rle with old method : 0.0397944450378418 time for calcul the mask position with numpy : 0.027556657791137695 nb_pixel_total : 51193 time to create 1 rle with old method : 0.05406069755554199 time for calcul the mask position with numpy : 0.028456926345825195 nb_pixel_total : 49762 time to create 1 rle with old method : 0.053046226501464844 time for calcul the mask position with numpy : 0.028223514556884766 nb_pixel_total : 5209 time to create 1 rle with old method : 0.005568265914916992 time for calcul the mask position with numpy : 0.028001070022583008 nb_pixel_total : 6528 time to create 1 rle with old method : 0.007204771041870117 time for calcul the mask position with numpy : 0.028027057647705078 nb_pixel_total : 24387 time to create 1 rle with old method : 0.02649712562561035 time for calcul the mask position with numpy : 0.027742385864257812 nb_pixel_total : 5764 time to create 1 rle with old method : 0.006224393844604492 create new chi : 4.20981240272522 time to delete rle : 0.0023827552795410156 batch 1 Loaded 71 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 16388 TO DO : save crop sub photo not yet done ! save time : 1.0501112937927246 nb_obj : 16 nb_hashtags : 3 time to prepare the origin masks : 10.34068489074707 time for calcul the mask position with numpy : 0.9721512794494629 nb_pixel_total : 6651835 time to create 1 rle with new method : 1.426469087600708 time for calcul the mask position with numpy : 0.027711868286132812 nb_pixel_total : 14375 time to create 1 rle with old method : 0.014523983001708984 time for calcul the mask position with numpy : 0.020323514938354492 nb_pixel_total : 25358 time to create 1 rle with old method : 0.027208566665649414 time for calcul the mask position with numpy : 0.020343780517578125 nb_pixel_total : 13415 time to create 1 rle with old method : 0.014232397079467773 time for calcul the mask position with numpy : 0.020356416702270508 nb_pixel_total : 5940 time to create 1 rle with old method : 0.0065860748291015625 time for calcul the mask position with numpy : 0.021215200424194336 nb_pixel_total : 22087 time to create 1 rle with old method : 0.023740768432617188 time for calcul the mask position with numpy : 0.0203402042388916 nb_pixel_total : 15645 time to create 1 rle with old method : 0.0171358585357666 time for calcul the mask position with numpy : 0.022016525268554688 nb_pixel_total : 55879 time to create 1 rle with old method : 0.05898284912109375 time for calcul the mask position with numpy : 0.024385690689086914 nb_pixel_total : 22107 time to create 1 rle with old method : 0.023677349090576172 time for calcul the mask position with numpy : 0.020641565322875977 nb_pixel_total : 34439 time to create 1 rle with old method : 0.03620266914367676 time for calcul the mask position with numpy : 0.019567251205444336 nb_pixel_total : 21299 time to create 1 rle with old method : 0.023154258728027344 time for calcul the mask position with numpy : 0.03251981735229492 nb_pixel_total : 25520 time to create 1 rle with old method : 0.026531219482421875 time for calcul the mask position with numpy : 0.03359794616699219 nb_pixel_total : 21884 time to create 1 rle with old method : 0.024198293685913086 time for calcul the mask position with numpy : 0.03321671485900879 nb_pixel_total : 10765 time to create 1 rle with old method : 0.011668682098388672 time for calcul the mask position with numpy : 0.033080339431762695 nb_pixel_total : 11782 time to create 1 rle with old method : 0.012933969497680664 time for calcul the mask position with numpy : 0.022684812545776367 nb_pixel_total : 72363 time to create 1 rle with old method : 0.07681703567504883 time for calcul the mask position with numpy : 0.021315813064575195 nb_pixel_total : 25547 time to create 1 rle with old method : 0.027181625366210938 create new chi : 3.248882532119751 time to delete rle : 0.0013358592987060547 batch 1 Loaded 33 chid ids of type : 3594 +++++++++++++++++++++++++Number RLEs to save : 8932 TO DO : save crop sub photo not yet done ! save time : 0.5539522171020508 nb_obj : 15 nb_hashtags : 3 time to prepare the origin masks : 8.933319807052612 time for calcul the mask position with numpy : 1.907182216644287 nb_pixel_total : 5970041 time to create 1 rle with new method : 0.7820613384246826 time for calcul the mask position with numpy : 0.032364845275878906 nb_pixel_total : 93083 time to create 1 rle with old method : 0.0989077091217041 time for calcul the mask position with numpy : 0.02048468589782715 nb_pixel_total : 28498 time to create 1 rle with old method : 0.02962040901184082 time for calcul the mask position with numpy : 0.019827604293823242 nb_pixel_total : 13277 time to create 1 rle with old method : 0.01668381690979004 time for calcul the mask position with numpy : 0.020441055297851562 nb_pixel_total : 77258 time to create 1 rle with old method : 0.0815732479095459 time for calcul the mask position with numpy : 0.02335524559020996 nb_pixel_total : 182333 time to create 1 rle with new method : 2.1814565658569336 time for calcul the mask position with numpy : 0.021805524826049805 nb_pixel_total : 45543 time to create 1 rle with old method : 0.049373626708984375 time for calcul the mask position with numpy : 0.021567583084106445 nb_pixel_total : 194159 time to create 1 rle with new method : 0.9716644287109375 time for calcul the mask position with numpy : 0.02428722381591797 nb_pixel_total : 31423 time to create 1 rle with old method : 0.03815460205078125 time for calcul the mask position with numpy : 0.02093648910522461 nb_pixel_total : 9474 time to create 1 rle with old method : 0.010243892669677734 time for calcul the mask position with numpy : 0.02307891845703125 nb_pixel_total : 244941 time to create 1 rle with new method : 1.190809965133667 time for calcul the mask position with numpy : 0.021470308303833008 nb_pixel_total : 83959 time to create 1 rle with old method : 0.09088134765625 time for calcul the mask position with numpy : 0.023819923400878906 nb_pixel_total : 8591 time to create 1 rle with old method : 0.009510278701782227 time for calcul the mask position with numpy : 0.021759033203125 nb_pixel_total : 13740 time to create 1 rle with old method : 0.015129566192626953 time for calcul the mask position with numpy : 0.021887540817260742 nb_pixel_total : 22758 time to create 1 rle with old method : 0.03098154067993164 time for calcul the mask position with numpy : 0.023601770401000977 nb_pixel_total : 31162 time to create 1 rle with old method : 0.03404951095581055 create new chi : 7.9907355308532715 time to delete rle : 0.0025975704193115234 batch 1 Loaded 31 chid ids of type : 3594 ++++++++++++++++++++Number RLEs to save : 11731 TO DO : save crop sub photo not yet done ! save time : 0.70735764503479 map_output_result : {1359972916: (0.0, 'Should be the crop_list due to order', 0), 1359972904: (0.0, 'Should be the crop_list due to order', 0), 1359972890: (0.0, 'Should be the crop_list due to order', 0), 1359972869: (0.0, 'Should be the crop_list due to order', 0), 1359972863: (0.0, 'Should be the crop_list due to order', 0), 1359972811: (0.0, 'Should be the crop_list due to order', 0), 1359972808: (0.0, 'Should be the crop_list due to order', 0), 1359972805: (0.0, 'Should be the crop_list due to order', 0), 1359972683: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1359972916, 1359972904, 1359972890, 1359972869, 1359972863, 1359972811, 1359972808, 1359972805, 1359972683] Looping around the photos to save general results len do output : 9 /1359972916.Didn't retrieve data . /1359972904.Didn't retrieve data . /1359972890.Didn't retrieve data . /1359972869.Didn't retrieve data . /1359972863.Didn't retrieve data . /1359972811.Didn't retrieve data . /1359972808.Didn't retrieve data . /1359972805.Didn't retrieve data . /1359972683.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972916', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972904', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972890', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972869', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972863', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972811', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972808', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972805', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972683', None, None, None, None, None, '2927940') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 27 time used for this insertion : 0.012819766998291016 save_final save missing photos in datou_result : time spend for datou_step_exec : 131.5211250782013 time spend to save output : 0.01331782341003418 total time spend for step 3 : 131.53444290161133 step4:ventilate_hashtags_in_portfolio Wed May 21 19:05:18 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 23219689 get user id for portfolio 23219689 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23219689 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('flou','pehd','mal_croppe','metal','autre','carton','pet_clair','background','environnement','papier','pet_fonce')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23219689 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('flou','pehd','mal_croppe','metal','autre','carton','pet_clair','background','environnement','papier','pet_fonce')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23219689 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('flou','pehd','mal_croppe','metal','autre','carton','pet_clair','background','environnement','papier','pet_fonce')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/23222729,23222730,23222731,23222732,23222733,23222734,23222735,23222736,23222737,23222738,23222739?tags=flou,pehd,mal_croppe,metal,autre,carton,pet_clair,background,environnement,papier,pet_fonce Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1359972916, 1359972904, 1359972890, 1359972869, 1359972863, 1359972811, 1359972808, 1359972805, 1359972683] Looping around the photos to save general results len do output : 1 /23219689. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972916', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972904', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972890', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972869', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972863', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972811', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972808', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972805', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972683', None, None, None, None, None, '2927940') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.013578414916992188 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.544935703277588 time spend to save output : 0.013828277587890625 total time spend for step 4 : 1.5587639808654785 step5:final Wed May 21 19:05:20 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1359972916: ('0.12381409868474136',), 1359972904: ('0.12381409868474136',), 1359972890: ('0.12381409868474136',), 1359972869: ('0.12381409868474136',), 1359972863: ('0.12381409868474136',), 1359972811: ('0.12381409868474136',), 1359972808: ('0.12381409868474136',), 1359972805: ('0.12381409868474136',), 1359972683: ('0.12381409868474136',)} new output for save of step final : {1359972916: ('0.12381409868474136',), 1359972904: ('0.12381409868474136',), 1359972890: ('0.12381409868474136',), 1359972869: ('0.12381409868474136',), 1359972863: ('0.12381409868474136',), 1359972811: ('0.12381409868474136',), 1359972808: ('0.12381409868474136',), 1359972805: ('0.12381409868474136',), 1359972683: ('0.12381409868474136',)} [1359972916, 1359972904, 1359972890, 1359972869, 1359972863, 1359972811, 1359972808, 1359972805, 1359972683] Looping around the photos to save general results len do output : 9 /1359972916.Didn't retrieve data . /1359972904.Didn't retrieve data . /1359972890.Didn't retrieve data . /1359972869.Didn't retrieve data . /1359972863.Didn't retrieve data . /1359972811.Didn't retrieve data . /1359972808.Didn't retrieve data . /1359972805.Didn't retrieve data . /1359972683.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972916', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972904', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972890', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972869', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972863', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972811', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972808', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972805', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972683', None, None, None, None, None, '2927940') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 27 time used for this insertion : 0.013664484024047852 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.11813926696777344 time spend to save output : 0.041853904724121094 total time spend for step 5 : 0.15999317169189453 step6:blur_detection Wed May 21 19:05:20 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748.jpg resize: (2160, 3264) 1359972916 -5.669739320475359 treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1.jpg resize: (2160, 3264) 1359972904 -2.4872603703181206 treat image : temp/1747846828_3880609_1359972890_59de8265ddb391b69e79def4c523a63a.jpg resize: (2160, 3264) 1359972890 -0.7078222709773337 treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9.jpg resize: (2160, 3264) 1359972869 -1.2209694159412776 treat image : temp/1747846828_3880609_1359972863_2f02e4521b09466bf7c24958df079d2b.jpg resize: (2160, 3264) 1359972863 -0.3330187440114261 treat image : temp/1747846828_3880609_1359972811_b4b4ba003a201f97dbf7e6da97be2399.jpg resize: (2160, 3264) 1359972811 0.6346238150771938 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e.jpg resize: (2160, 3264) 1359972808 -3.496607434127563 treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2.jpg resize: (2160, 3264) 1359972805 -0.08476029360496651 treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad.jpg resize: (2160, 3264) 1359972683 -4.7382477749174 treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002368_0.png resize: (341, 169) 1360114105 -1.8702938101887334 treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002377_0.png resize: (134, 117) 1360114107 -1.579747074505329 treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002374_0.png resize: (287, 237) 1360114108 -2.915536405205823 treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002380_0.png resize: (276, 302) 1360114109 -1.3728395480646127 treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002372_0.png resize: (436, 543) 1360114111 -4.097173164333327 treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002379_0.png resize: (162, 322) 1360114112 -2.3508070117543443 treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002376_0.png resize: (188, 225) 1360114113 -3.5184501590131956 treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002366_0.png resize: (275, 287) 1360114115 -4.6740095908769685 treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002365_0.png resize: (201, 178) 1360114116 -1.6668485681364573 treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002369_0.png resize: (128, 147) 1360114117 -1.9368622914612947 treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002371_0.png resize: (207, 247) 1360114119 -1.6077091398030319 treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002363_0.png resize: (173, 199) 1360114120 -0.6947402905065767 treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002382_0.png resize: (96, 202) 1360114121 -1.8105849313411362 treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002370_0.png resize: (366, 237) 1360114123 -1.9531717264112922 treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002384_0.png resize: (217, 408) 1360114124 -1.3365264172903615 treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002390_0.png resize: (477, 822) 1360114125 -1.6698859726816764 treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002393_0.png resize: (155, 283) 1360114127 -2.24868317929346 treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002389_0.png resize: (277, 218) 1360114128 -2.1048884383079165 treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002394_0.png resize: (204, 162) 1360114129 -1.025590052741935 treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002388_0.png resize: (308, 221) 1360114132 -1.0324556538293679 treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002391_0.png resize: (247, 87) 1360114133 -2.0931509118584137 treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002399_0.png resize: (216, 88) 1360114134 -0.7698128846115395 treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002386_0.png resize: (318, 221) 1360114136 -1.4989217374406758 treat image : temp/1747846828_3880609_1359972890_59de8265ddb391b69e79def4c523a63a_rle_crop_3808002401_0.png resize: (224, 323) 1360114137 -1.2318668064863376 treat image : temp/1747846828_3880609_1359972890_59de8265ddb391b69e79def4c523a63a_rle_crop_3808002411_0.png resize: (476, 316) 1360114138 -1.9392855140553833 treat image : temp/1747846828_3880609_1359972890_59de8265ddb391b69e79def4c523a63a_rle_crop_3808002405_0.png resize: (193, 100) 1360114140 -1.2491871844395417 treat image : temp/1747846828_3880609_1359972890_59de8265ddb391b69e79def4c523a63a_rle_crop_3808002408_0.png resize: (142, 141) 1360114141 0.4685306936925029 treat image : temp/1747846828_3880609_1359972890_59de8265ddb391b69e79def4c523a63a_rle_crop_3808002412_0.png resize: (261, 234) 1360114142 -1.2938598780597423 treat image : temp/1747846828_3880609_1359972890_59de8265ddb391b69e79def4c523a63a_rle_crop_3808002406_0.png resize: (270, 185) 1360114144 -1.2109548116401432 treat image : temp/1747846828_3880609_1359972890_59de8265ddb391b69e79def4c523a63a_rle_crop_3808002410_0.png resize: (361, 429) 1360114145 -1.713731840103698 treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9_rle_crop_3808002425_0.png resize: (575, 229) 1360114146 -0.939055265679342 treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9_rle_crop_3808002417_0.png resize: (159, 220) 1360114147 -1.1566978091894462 treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9_rle_crop_3808002421_0.png resize: (201, 195) 1360114149 -1.5281676083380606 treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9_rle_crop_3808002416_0.png resize: (213, 182) 1360114150 -0.3527466782842052 treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9_rle_crop_3808002420_0.png resize: (218, 145) 1360114151 0.4825671698239061 treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9_rle_crop_3808002424_0.png resize: (253, 130) 1360114153 0.11941464915331655 treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9_rle_crop_3808002426_0.png resize: (394, 477) 1360114155 -2.2171010934505 treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9_rle_crop_3808002419_0.png resize: (428, 678) 1360114156 -2.4919945705927833 treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9_rle_crop_3808002415_0.png resize: (201, 269) 1360114158 -1.051202161652629 treat image : temp/1747846828_3880609_1359972863_2f02e4521b09466bf7c24958df079d2b_rle_crop_3808002430_0.png resize: (169, 342) 1360114160 0.12437565435605893 treat image : temp/1747846828_3880609_1359972863_2f02e4521b09466bf7c24958df079d2b_rle_crop_3808002433_0.png resize: (494, 607) 1360114161 -0.3548939629912727 treat image : temp/1747846828_3880609_1359972863_2f02e4521b09466bf7c24958df079d2b_rle_crop_3808002431_0.png resize: (285, 344) 1360114163 -0.39847258351180903 treat image : temp/1747846828_3880609_1359972811_b4b4ba003a201f97dbf7e6da97be2399_rle_crop_3808002442_0.png resize: (502, 224) 1360114165 -0.9267592419503025 treat image : temp/1747846828_3880609_1359972811_b4b4ba003a201f97dbf7e6da97be2399_rle_crop_3808002436_0.png resize: (867, 668) 1360114166 -0.5344536333971742 treat image : temp/1747846828_3880609_1359972811_b4b4ba003a201f97dbf7e6da97be2399_rle_crop_3808002443_0.png resize: (123, 183) 1360114168 -1.0113481334464234 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002460_0.png resize: (145, 58) 1360114170 -1.9793903977010974 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002444_0.png resize: (164, 190) 1360114171 -0.8137224428782387 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002453_0.png resize: (293, 82) 1360114173 -2.2452143695220474 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002447_0.png resize: (273, 216) 1360114175 -1.5937162459952487 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002449_0.png resize: (69, 108) 1360114176 0.2675798830620771 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002476_0.png resize: (184, 206) 1360114178 -1.8565202575993935 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002452_0.png resize: (183, 266) 1360114180 -1.2362517747119746 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002466_0.png resize: (119, 65) 1360114181 2.156916626041332 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002450_0.png resize: (145, 207) 1360114184 -1.661466382854344 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002469_0.png resize: (142, 229) 1360114185 -1.8409322018813812 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002448_0.png resize: (249, 364) 1360114187 -1.6234379412806712 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002454_0.png resize: (122, 145) 1360114189 -1.1751700668752283 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002451_0.png resize: (230, 142) 1360114190 -1.587034786498938 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002456_0.png resize: (205, 86) 1360114192 -2.484435694914568 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002458_0.png resize: (148, 295) 1360114194 -0.487738495039988 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002474_0.png resize: (205, 119) 1360114195 -1.8929907220968891 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002446_0.png resize: (228, 223) 1360114197 -0.8260787887408312 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002462_0.png resize: (267, 278) 1360114199 -3.3055643385564286 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002471_0.png resize: (196, 472) 1360114200 -0.25423478645675923 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002465_0.png resize: (161, 382) 1360114202 -1.0120500057225914 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002461_0.png resize: (156, 123) 1360114204 -1.6138813947628623 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002464_0.png resize: (116, 58) 1360114205 -1.4329568589966928 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002468_0.png resize: (249, 193) 1360114207 -2.4150859600481356 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002445_0.png resize: (90, 119) 1360114209 -2.622111771312334 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002457_0.png resize: (116, 204) 1360114210 -2.129597114240576 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002475_0.png resize: (137, 70) 1360114212 -0.4298194966441816 treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002493_0.png resize: (163, 196) 1360114214 0.5341141880587525 treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002489_0.png resize: (304, 200) 1360114215 -1.0476769500449425 treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002491_0.png resize: (114, 110) 1360114217 2.370953027299763 treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002481_0.png resize: (116, 120) 1360114219 3.4070050520972472 treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002494_0.png resize: (143, 146) 1360114220 -0.25492620692154455 treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002479_0.png resize: (203, 203) 1360114222 -0.05740915112775565 treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002487_0.png resize: (189, 209) 1360114224 -0.4400212670309295 treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002486_0.png resize: (159, 376) 1360114225 -0.11908313430228747 treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002480_0.png resize: (327, 353) 1360114227 -0.4848930771759642 treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002484_0.png resize: (203, 221) 1360114229 0.33309119533840253 treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002490_0.png resize: (245, 224) 1360114230 -0.5533376149945108 treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad_rle_crop_3808002509_0.png resize: (481, 321) 1360114231 -3.488384243409656 treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad_rle_crop_3808002498_0.png resize: (99, 112) 1360114232 -1.5516368285576776 treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad_rle_crop_3808002497_0.png resize: (120, 154) 1360114233 -2.82993482670477 treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad_rle_crop_3808002495_0.png resize: (200, 276) 1360114234 -3.2202989064373653 treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad_rle_crop_3808002507_0.png resize: (150, 130) 1360114235 -2.011556652035925 treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad_rle_crop_3808002508_0.png resize: (223, 253) 1360114236 -3.2680439242448545 treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad_rle_crop_3808002501_0.png resize: (160, 94) 1360114237 -2.89659404348697 treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002364_0.png resize: (164, 171) 1360114240 -1.2832252892375287 treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002381_0.png resize: (146, 290) 1360114241 -0.6525259180583096 treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002385_0.png resize: (133, 294) 1360114242 -0.6635528703670156 treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002398_0.png resize: (141, 105) 1360114243 0.3011706686776216 treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002387_0.png resize: (89, 119) 1360114244 -2.4090999439567318 treat image : temp/1747846828_3880609_1359972890_59de8265ddb391b69e79def4c523a63a_rle_crop_3808002414_0.png resize: (234, 137) 1360114245 -2.165709253956037 treat image : temp/1747846828_3880609_1359972890_59de8265ddb391b69e79def4c523a63a_rle_crop_3808002404_0.png resize: (168, 193) 1360114246 -1.4315498183742532 treat image : temp/1747846828_3880609_1359972890_59de8265ddb391b69e79def4c523a63a_rle_crop_3808002403_0.png resize: (192, 313) 1360114247 -0.976411222075847 treat image : temp/1747846828_3880609_1359972890_59de8265ddb391b69e79def4c523a63a_rle_crop_3808002409_0.png resize: (767, 1149) 1360114248 0.4857025227291652 treat image : temp/1747846828_3880609_1359972890_59de8265ddb391b69e79def4c523a63a_rle_crop_3808002402_0.png resize: (536, 531) 1360114249 -0.8246837811661241 treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9_rle_crop_3808002418_0.png resize: (170, 306) 1360114250 -1.8974252587679532 treat image : temp/1747846828_3880609_1359972811_b4b4ba003a201f97dbf7e6da97be2399_rle_crop_3808002434_0.png resize: (186, 168) 1360114251 -0.0807539642920552 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002459_0.png resize: (151, 173) 1360114252 -1.2546303052996957 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002467_0.png resize: (164, 149) 1360114253 -2.093915234577119 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002478_0.png resize: (96, 164) 1360114254 -1.2394247932563445 treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002482_0.png resize: (121, 128) 1360114255 -0.8807419494729339 treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002492_0.png resize: (146, 123) 1360114256 -0.09074901786120576 treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002483_0.png resize: (227, 169) 1360114257 -0.657051862862514 treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002488_0.png resize: (255, 318) 1360114258 -0.13474726992167366 treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad_rle_crop_3808002496_0.png resize: (191, 158) 1360114259 -1.224345799609233 treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad_rle_crop_3808002503_0.png resize: (562, 582) 1360114260 -3.002721476408659 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002463_0.png resize: (138, 171) 1360114271 -1.8966249362643415 treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002373_0.png resize: (222, 353) 1360114516 -2.7448375707494437 treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002378_0.png resize: (254, 487) 1360114518 -2.323211401766255 treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002367_0.png resize: (147, 386) 1360114520 -3.9083670948166302 treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002375_0.png resize: (210, 424) 1360114521 -3.373730652656956 treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002395_0.png resize: (267, 289) 1360114523 -3.299298167996309 treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002397_0.png resize: (220, 362) 1360114525 -2.3096119258223275 treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002383_0.png resize: (288, 288) 1360114526 -2.3939985288296772 treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002392_0.png resize: (249, 378) 1360114528 -2.159682541719182 treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002400_0.png resize: (420, 245) 1360114530 -1.3772282724203673 treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002396_0.png resize: (190, 225) 1360114531 -2.7550538548982666 treat image : temp/1747846828_3880609_1359972890_59de8265ddb391b69e79def4c523a63a_rle_crop_3808002407_0.png resize: (159, 375) 1360114534 -0.6705655832794452 treat image : temp/1747846828_3880609_1359972890_59de8265ddb391b69e79def4c523a63a_rle_crop_3808002413_0.png resize: (261, 395) 1360114536 -2.3984616401563805 treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9_rle_crop_3808002423_0.png resize: (186, 157) 1360114537 0.05635930276049115 treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9_rle_crop_3808002428_0.png resize: (187, 202) 1360114539 -2.0541424026837105 treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9_rle_crop_3808002429_0.png resize: (401, 297) 1360114541 -0.9114835423250552 treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9_rle_crop_3808002427_0.png resize: (348, 181) 1360114542 -0.7918496249301402 treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9_rle_crop_3808002422_0.png resize: (739, 412) 1360114544 -0.852672321602557 treat image : temp/1747846828_3880609_1359972863_2f02e4521b09466bf7c24958df079d2b_rle_crop_3808002432_0.png resize: (440, 729) 1360114546 -0.8184394717587319 treat image : temp/1747846828_3880609_1359972811_b4b4ba003a201f97dbf7e6da97be2399_rle_crop_3808002435_0.png resize: (323, 401) 1360114547 -1.0694306459260878 treat image : temp/1747846828_3880609_1359972811_b4b4ba003a201f97dbf7e6da97be2399_rle_crop_3808002438_0.png resize: (202, 166) 1360114549 0.169079056277119 treat image : temp/1747846828_3880609_1359972811_b4b4ba003a201f97dbf7e6da97be2399_rle_crop_3808002441_0.png resize: (235, 303) 1360114551 -1.0228575586135245 treat image : temp/1747846828_3880609_1359972811_b4b4ba003a201f97dbf7e6da97be2399_rle_crop_3808002440_0.png resize: (135, 232) 1360114552 0.9280828035585151 treat image : temp/1747846828_3880609_1359972811_b4b4ba003a201f97dbf7e6da97be2399_rle_crop_3808002437_0.png resize: (504, 586) 1360114554 -0.43622725218448344 treat image : temp/1747846828_3880609_1359972811_b4b4ba003a201f97dbf7e6da97be2399_rle_crop_3808002439_0.png resize: (476, 357) 1360114556 -1.339058656586359 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002477_0.png resize: (182, 99) 1360114557 -2.691497844294809 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002455_0.png resize: (170, 202) 1360114560 -2.9975589864424057 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002473_0.png resize: (580, 537) 1360114562 -3.5475488096773455 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002472_0.png resize: (288, 281) 1360114563 -3.763019140781739 treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002470_0.png resize: (268, 275) 1360114565 -3.68054542864081 treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002485_0.png resize: (203, 245) 1360114567 -2.512308786193749 treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad_rle_crop_3808002500_0.png resize: (629, 583) 1360114568 -2.513258098663751 treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad_rle_crop_3808002504_0.png resize: (260, 235) 1360114570 -2.3669419139238443 treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad_rle_crop_3808002499_0.png resize: (148, 741) 1360114572 -0.6958554753622579 treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad_rle_crop_3808002506_0.png resize: (312, 407) 1360114573 -3.697804874480367 treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad_rle_crop_3808002505_0.png resize: (729, 460) 1360114575 -4.141538951623917 treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad_rle_crop_3808002502_0.png resize: (156, 248) 1360114577 -1.7091083310319763 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 156 time used for this insertion : 0.019223690032958984 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 156 time used for this insertion : 0.031996965408325195 save missing photos in datou_result : time spend for datou_step_exec : 38.968185901641846 time spend to save output : 0.05709433555603027 total time spend for step 6 : 39.025280237197876 step7:brightness Wed May 21 19:05:59 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748.jpg treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1.jpg treat image : temp/1747846828_3880609_1359972890_59de8265ddb391b69e79def4c523a63a.jpg treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9.jpg treat image : temp/1747846828_3880609_1359972863_2f02e4521b09466bf7c24958df079d2b.jpg treat image : temp/1747846828_3880609_1359972811_b4b4ba003a201f97dbf7e6da97be2399.jpg treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e.jpg treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2.jpg treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad.jpg treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002368_0.png treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002377_0.png treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002374_0.png treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002380_0.png treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002372_0.png treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002379_0.png treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002376_0.png treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002366_0.png treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002365_0.png treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002369_0.png treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002371_0.png treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002363_0.png treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002382_0.png treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002370_0.png treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002384_0.png treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002390_0.png treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002393_0.png treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002389_0.png treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002394_0.png treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002388_0.png treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002391_0.png treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002399_0.png treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002386_0.png treat image : temp/1747846828_3880609_1359972890_59de8265ddb391b69e79def4c523a63a_rle_crop_3808002401_0.png treat image : temp/1747846828_3880609_1359972890_59de8265ddb391b69e79def4c523a63a_rle_crop_3808002411_0.png treat image : temp/1747846828_3880609_1359972890_59de8265ddb391b69e79def4c523a63a_rle_crop_3808002405_0.png treat image : temp/1747846828_3880609_1359972890_59de8265ddb391b69e79def4c523a63a_rle_crop_3808002408_0.png treat image : temp/1747846828_3880609_1359972890_59de8265ddb391b69e79def4c523a63a_rle_crop_3808002412_0.png treat image : temp/1747846828_3880609_1359972890_59de8265ddb391b69e79def4c523a63a_rle_crop_3808002406_0.png treat image : temp/1747846828_3880609_1359972890_59de8265ddb391b69e79def4c523a63a_rle_crop_3808002410_0.png treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9_rle_crop_3808002425_0.png treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9_rle_crop_3808002417_0.png treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9_rle_crop_3808002421_0.png treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9_rle_crop_3808002416_0.png treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9_rle_crop_3808002420_0.png treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9_rle_crop_3808002424_0.png treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9_rle_crop_3808002426_0.png treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9_rle_crop_3808002419_0.png treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9_rle_crop_3808002415_0.png treat image : temp/1747846828_3880609_1359972863_2f02e4521b09466bf7c24958df079d2b_rle_crop_3808002430_0.png treat image : temp/1747846828_3880609_1359972863_2f02e4521b09466bf7c24958df079d2b_rle_crop_3808002433_0.png treat image : temp/1747846828_3880609_1359972863_2f02e4521b09466bf7c24958df079d2b_rle_crop_3808002431_0.png treat image : temp/1747846828_3880609_1359972811_b4b4ba003a201f97dbf7e6da97be2399_rle_crop_3808002442_0.png treat image : temp/1747846828_3880609_1359972811_b4b4ba003a201f97dbf7e6da97be2399_rle_crop_3808002436_0.png treat image : temp/1747846828_3880609_1359972811_b4b4ba003a201f97dbf7e6da97be2399_rle_crop_3808002443_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002460_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002444_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002453_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002447_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002449_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002476_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002452_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002466_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002450_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002469_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002448_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002454_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002451_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002456_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002458_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002474_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002446_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002462_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002471_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002465_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002461_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002464_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002468_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002445_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002457_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002475_0.png treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002493_0.png treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002489_0.png treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002491_0.png treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002481_0.png treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002494_0.png treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002479_0.png treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002487_0.png treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002486_0.png treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002480_0.png treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002484_0.png treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002490_0.png treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad_rle_crop_3808002509_0.png treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad_rle_crop_3808002498_0.png treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad_rle_crop_3808002497_0.png treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad_rle_crop_3808002495_0.png treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad_rle_crop_3808002507_0.png treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad_rle_crop_3808002508_0.png treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad_rle_crop_3808002501_0.png treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002364_0.png treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002381_0.png treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002385_0.png treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002398_0.png treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002387_0.png treat image : temp/1747846828_3880609_1359972890_59de8265ddb391b69e79def4c523a63a_rle_crop_3808002414_0.png treat image : temp/1747846828_3880609_1359972890_59de8265ddb391b69e79def4c523a63a_rle_crop_3808002404_0.png treat image : temp/1747846828_3880609_1359972890_59de8265ddb391b69e79def4c523a63a_rle_crop_3808002403_0.png treat image : temp/1747846828_3880609_1359972890_59de8265ddb391b69e79def4c523a63a_rle_crop_3808002409_0.png treat image : temp/1747846828_3880609_1359972890_59de8265ddb391b69e79def4c523a63a_rle_crop_3808002402_0.png treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9_rle_crop_3808002418_0.png treat image : temp/1747846828_3880609_1359972811_b4b4ba003a201f97dbf7e6da97be2399_rle_crop_3808002434_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002459_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002467_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002478_0.png treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002482_0.png treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002492_0.png treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002483_0.png treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002488_0.png treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad_rle_crop_3808002496_0.png treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad_rle_crop_3808002503_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002463_0.png treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002373_0.png treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002378_0.png treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002367_0.png treat image : temp/1747846828_3880609_1359972916_a47ab880c79bcf04bf215e100d704748_rle_crop_3808002375_0.png treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002395_0.png treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002397_0.png treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002383_0.png treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002392_0.png treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002400_0.png treat image : temp/1747846828_3880609_1359972904_24465f59ee90ace2a20b56374f428ae1_rle_crop_3808002396_0.png treat image : temp/1747846828_3880609_1359972890_59de8265ddb391b69e79def4c523a63a_rle_crop_3808002407_0.png treat image : temp/1747846828_3880609_1359972890_59de8265ddb391b69e79def4c523a63a_rle_crop_3808002413_0.png treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9_rle_crop_3808002423_0.png treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9_rle_crop_3808002428_0.png treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9_rle_crop_3808002429_0.png treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9_rle_crop_3808002427_0.png treat image : temp/1747846828_3880609_1359972869_0fdd0e67c8e39d46c1c50e5fe09076a9_rle_crop_3808002422_0.png treat image : temp/1747846828_3880609_1359972863_2f02e4521b09466bf7c24958df079d2b_rle_crop_3808002432_0.png treat image : temp/1747846828_3880609_1359972811_b4b4ba003a201f97dbf7e6da97be2399_rle_crop_3808002435_0.png treat image : temp/1747846828_3880609_1359972811_b4b4ba003a201f97dbf7e6da97be2399_rle_crop_3808002438_0.png treat image : temp/1747846828_3880609_1359972811_b4b4ba003a201f97dbf7e6da97be2399_rle_crop_3808002441_0.png treat image : temp/1747846828_3880609_1359972811_b4b4ba003a201f97dbf7e6da97be2399_rle_crop_3808002440_0.png treat image : temp/1747846828_3880609_1359972811_b4b4ba003a201f97dbf7e6da97be2399_rle_crop_3808002437_0.png treat image : temp/1747846828_3880609_1359972811_b4b4ba003a201f97dbf7e6da97be2399_rle_crop_3808002439_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002477_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002455_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002473_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002472_0.png treat image : temp/1747846828_3880609_1359972808_3fc22acff82200f907bc8c5ec76a577e_rle_crop_3808002470_0.png treat image : temp/1747846828_3880609_1359972805_9c8d4122967227195a617f487aca8ba2_rle_crop_3808002485_0.png treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad_rle_crop_3808002500_0.png treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad_rle_crop_3808002504_0.png treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad_rle_crop_3808002499_0.png treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad_rle_crop_3808002506_0.png treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad_rle_crop_3808002505_0.png treat image : temp/1747846828_3880609_1359972683_8ef08b656a39c94a8a9a184394e50fad_rle_crop_3808002502_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 156 time used for this insertion : 0.02093791961669922 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 156 time used for this insertion : 0.036406517028808594 save missing photos in datou_result : time spend for datou_step_exec : 9.99242639541626 time spend to save output : 0.06277632713317871 total time spend for step 7 : 10.055202722549438 step8:velours_tree Wed May 21 19:06:09 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.2850029468536377 time spend to save output : 4.863739013671875e-05 total time spend for step 8 : 0.2850515842437744 step9:send_mail_cod Wed May 21 19:06:09 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P23219689_21-05-2025_19_06_09.pdf 23222729 imagette232227291747847169 23222730 imagette232227301747847169 23222731 imagette232227311747847169 23222732 change filename to text .imagette232227321747847169 23222733 imagette232227331747847169 23222734 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette232227341747847169 23222735 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette232227351747847171 23222736 imagette232227361747847172 23222738 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette232227381747847172 23222739 imagette232227391747847174 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=23219689 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/23222729,23222730,23222731,23222732,23222733,23222734,23222735,23222736,23222737,23222738,23222739?tags=flou,pehd,mal_croppe,metal,autre,carton,pet_clair,background,environnement,papier,pet_fonce args[1359972916] : ((1359972916, -5.669739320475359, 492609224), (1359972916, 0.3765899085109922, 2107752395), '0.12381409868474136') We are sending mail with results at report@fotonower.com args[1359972904] : ((1359972904, -2.4872603703181206, 492609224), (1359972904, 0.1938221476618695, 2107752395), '0.12381409868474136') We are sending mail with results at report@fotonower.com args[1359972890] : ((1359972890, -0.7078222709773337, 492688767), (1359972890, -0.15220288688994882, 496442774), '0.12381409868474136') We are sending mail with results at report@fotonower.com args[1359972869] : ((1359972869, -1.2209694159412776, 492688767), (1359972869, 0.09086062505814199, 2107752395), '0.12381409868474136') We are sending mail with results at report@fotonower.com args[1359972863] : ((1359972863, -0.3330187440114261, 492688767), (1359972863, 0.12227160075452843, 2107752395), '0.12381409868474136') We are sending mail with results at report@fotonower.com args[1359972811] : ((1359972811, 0.6346238150771938, 492688767), (1359972811, 0.4039019355952065, 2107752395), '0.12381409868474136') We are sending mail with results at report@fotonower.com args[1359972808] : ((1359972808, -3.496607434127563, 492609224), (1359972808, 0.25481899973270855, 2107752395), '0.12381409868474136') We are sending mail with results at report@fotonower.com args[1359972805] : ((1359972805, -0.08476029360496651, 492688767), (1359972805, 0.373436023957387, 2107752395), '0.12381409868474136') We are sending mail with results at report@fotonower.com args[1359972683] : ((1359972683, -4.7382477749174, 492609224), (1359972683, 0.37977809967028575, 2107752395), '0.12381409868474136') We are sending mail with results at report@fotonower.com refus_total : 0.12381409868474136 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=23219689 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23219689_21-05-2025_19_06_09.pdf results_Auto_P23219689_21-05-2025_19_06_09.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23219689_21-05-2025_19_06_09.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','23219689','results_Auto_P23219689_21-05-2025_19_06_09.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23219689_21-05-2025_19_06_09.pdf','pdf','','0.61','0.12381409868474136') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/23219689

https://www.fotonower.com/image?json=false&list_photos_id=1359972916
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359972904
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359972890
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359972869
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359972863
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359972811
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359972808
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359972805
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359972683
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 12.38%
Veuillez trouver les photos des contaminants.

exemples de contaminants: metal: https://www.fotonower.com/view/23222732?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/23222734?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/23222735?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/23222738?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23219689_21-05-2025_19_06_09.pdf.

Lien vers velours :https://www.fotonower.com/velours/23222729,23222730,23222731,23222732,23222733,23222734,23222735,23222736,23222737,23222738,23222739?tags=flou,pehd,mal_croppe,metal,autre,carton,pet_clair,background,environnement,papier,pet_fonce.


L'équipe Fotonower 202 b'' Server: nginx Date: Wed, 21 May 2025 17:06:17 GMT Content-Length: 0 Connection: close X-Message-Id: Li2-JvK0Rki0uXWn8pthwA Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1359972916, 1359972904, 1359972890, 1359972869, 1359972863, 1359972811, 1359972808, 1359972805, 1359972683] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972916', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972904', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972890', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972869', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972863', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972811', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972808', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972805', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972683', None, None, None, None, None, '2927940') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.013248920440673828 save_final save missing photos in datou_result : time spend for datou_step_exec : 7.59655499458313 time spend to save output : 0.013451814651489258 total time spend for step 9 : 7.610006809234619 step10:split_time_score Wed May 21 19:06:17 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('07', 9),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 21052025 23219689 Nombre de photos uploadées : 9 / 23040 (0%) 21052025 23219689 Nombre de photos taguées (types de déchets): 0 / 9 (0%) 21052025 23219689 Nombre de photos taguées (volume) : 0 / 9 (0%) elapsed_time : load_data_split_time_score 1.6689300537109375e-06 elapsed_time : order_list_meta_photo_and_scores 5.4836273193359375e-06 ????????? elapsed_time : fill_and_build_computed_from_old_data 0.0005044937133789062 elapsed_time : insert_dashboard_record_day_entry 0.02390289306640625 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.08331856787854033 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23219688_21-05-2025_18_56_04.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23219688 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23219688 AND mptpi.`type`=3594 To do Qualite : 0.12381409868474136 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23219689_21-05-2025_19_06_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23219689 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23219689 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23219717 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23219758 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23219775 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23219778 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23219783 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23219791 order by id desc limit 1 Qualite : 0.1417862796160131 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23219795_21-05-2025_18_37_05.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23219795 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23219795 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23219802 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23219856 order by id desc limit 1 Qualite : 0.1421742583028582 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23219858_21-05-2025_18_30_33.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23219858 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23219858 AND mptpi.`type`=3594 To do Qualite : 0.09422273973246707 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23219879_21-05-2025_18_18_51.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23219879 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23219879 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23219881 order by id desc limit 1 NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'21052025': {'nb_upload': 9, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1359972916, 1359972904, 1359972890, 1359972869, 1359972863, 1359972811, 1359972808, 1359972805, 1359972683] Looping around the photos to save general results len do output : 1 /23219689Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972916', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972904', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972890', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972869', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972863', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972811', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972808', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972805', None, None, None, None, None, '2927940') ('3318', None, None, None, None, None, None, None, '2927940') ('3318', '23219689', '1359972683', None, None, None, None, None, '2927940') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.014282464981079102 save_final save missing photos in datou_result : time spend for datou_step_exec : 8.471721410751343 time spend to save output : 0.014478683471679688 total time spend for step 10 : 8.486200094223022 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 9 set_done_treatment 130.65user 171.48system 6:01.40elapsed 83%CPU (0avgtext+0avgdata 5326448maxresident)k 681072inputs+85888outputs (13358major+12818346minor)pagefaults 0swaps