python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 12 -a ' -a 533 -l 200 --rand=1 ' -s datou_current_533 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2420980 load datou : 533 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec no input labels no input values updating current state to 1 we have a portfolio with more photos than limit : 371>200 please execute split_portfolio.py -i 31145473 -l 200 size over we load limit photo not treated list_input_json: {} Date 2026-06-10 19:50:26.893023 Current got : datou_id : 533, datou_cur_ids : ['4385217'] with mtr_portfolio_ids : ['31145473'] and first list_photo_ids : [] new path : /proc/2420980/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! DataTypes for each output/input checked ! List Step Type Loaded in datou : tfhub_classification2, argmax list_input_json : {} origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 200 ; length of list_pids : 200 ; length of list_args : 200 time to download the photos : 46.392184257507324 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 2 step1:tfhub_classification2 Wed Jun 10 19:51:13 2026 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou_step TFHub with tf2 ! nombre de thcls : 3 we are using the classfication for multi_thcl [3513, 3379, 3847] begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 3096 l 3632 free memory gpu now : 1752 wait 20 seconds inside check gpu memory havn't enough memory gpu , need / 3096 l 3632 free memory gpu now : 70 wait 20 seconds inside check gpu memory havn't enough memory gpu , need / 3096 l 3632 free memory gpu now : 70 wait 20 seconds inside check gpu memory havn't enough memory gpu , need / 3096 l 3632 free memory gpu now : 70 wait 20 seconds inside check gpu memory havn't enough memory gpu , need / 3096 l 3632 free memory gpu now : 70 wait 20 seconds inside check gpu memory 2026-06-10 19:53:01.491321: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2026-06-10 19:53:01.523268: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2026-06-10 19:53:01.523528: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2026-06-10 19:53:01.525431: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2026-06-10 19:53:01.527311: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2026-06-10 19:53:01.527656: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2026-06-10 19:53:01.529648: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2026-06-10 19:53:01.530908: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2026-06-10 19:53:01.536251: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2026-06-10 19:53:01.537263: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2026-06-10 19:53:01.537687: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2026-06-10 19:53:01.570557: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493010000 Hz 2026-06-10 19:53:01.572673: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f0b84000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2026-06-10 19:53:01.572718: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2026-06-10 19:53:01.691106: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x14363ea0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2026-06-10 19:53:01.691149: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2026-06-10 19:53:01.692125: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2026-06-10 19:53:01.692215: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2026-06-10 19:53:01.692249: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2026-06-10 19:53:01.692288: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2026-06-10 19:53:01.692338: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2026-06-10 19:53:01.692391: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2026-06-10 19:53:01.692449: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2026-06-10 19:53:01.692486: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2026-06-10 19:53:01.693793: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2026-06-10 19:53:01.693865: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2026-06-10 19:53:01.694770: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2026-06-10 19:53:01.694798: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2026-06-10 19:53:01.694812: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2026-06-10 19:53:01.695934: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3096 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) l 3637 free memory gpu now : 2551 max_wait_temp : 6 max_wait : 5 1 Physical GPUs, 1 Logical GPUs tagging for thcl : 3513 To do loadFromThcl(), then load ParamDescType : thcl3513 thcls : [{'id': 3513, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2_tf', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 4557, 'photo_desc_type': 5767, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] thcl {'id': 3513, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2_tf', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 4557, 'photo_desc_type': 5767, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'} Update svm_hashtag_type_desc : 5767 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5767, 'Rungis_amount_dechets_fall_2018_v2_tf', 2048, 2048, 'Rungis_amount_dechets_fall_2018_v2_tf', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 3, 16, 15, 52, 10), datetime.datetime(2023, 3, 16, 15, 52, 10)) model_name : Rungis_amount_dechets_fall_2018_v2_tf model_param file didn't exist model_name : Rungis_amount_dechets_fall_2018_v2_tf model_type : tf_classification2 list file need : ['Confusion_Matrix.png', 'Precision_Recall_05102018_Papier_non_papier_dense.jpg', 'Precision_Recall_05102018_Papier_non_papier_peu_dense.jpg', 'Precision_Recall_05102018_Papier_non_papier_presque_vide.jpg', 'Precision_Recall_05102018_Papier_non_papier_tres_dense.jpg', 'Precision_Recall_05102018_Papier_non_papier_tres_peu_dense.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00001', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file exist in s3 : ['Confusion_Matrix.png', 'Precision_Recall_05102018_Papier_non_papier_dense.jpg', 'Precision_Recall_05102018_Papier_non_papier_peu_dense.jpg', 'Precision_Recall_05102018_Papier_non_papier_presque_vide.jpg', 'Precision_Recall_05102018_Papier_non_papier_tres_dense.jpg', 'Precision_Recall_05102018_Papier_non_papier_tres_peu_dense.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00001', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file manque in s3 : [] 2026-06-10 19:53:07.941259: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.02G (3246391296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2026-06-10 19:53:07.941821: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.72G (2921752064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2026-06-10 19:53:07.942402: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.45G (2629576704 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2026-06-10 19:53:07.942939: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.20G (2366618880 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/Confusion_Matrix.png size_local : 67810 size in s3 : 67810 create time local : 2026-03-04 13:31:58 create time in s3 : 2026-03-04 12:24:14 Confusion_Matrix.png already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/Precision_Recall_05102018_Papier_non_papier_dense.jpg size_local : 73949 size in s3 : 73949 create time local : 2026-03-04 13:31:58 create time in s3 : 2026-03-04 12:24:15 Precision_Recall_05102018_Papier_non_papier_dense.jpg already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/Precision_Recall_05102018_Papier_non_papier_peu_dense.jpg size_local : 85572 size in s3 : 85572 create time local : 2026-03-04 13:31:58 create time in s3 : 2026-03-04 12:24:15 Precision_Recall_05102018_Papier_non_papier_peu_dense.jpg already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/Precision_Recall_05102018_Papier_non_papier_presque_vide.jpg size_local : 72361 size in s3 : 72361 create time local : 2026-03-04 13:31:58 create time in s3 : 2026-03-04 12:24:16 Precision_Recall_05102018_Papier_non_papier_presque_vide.jpg already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/Precision_Recall_05102018_Papier_non_papier_tres_dense.jpg size_local : 83567 size in s3 : 83567 create time local : 2026-03-04 13:31:58 create time in s3 : 2026-03-04 12:24:16 Precision_Recall_05102018_Papier_non_papier_tres_dense.jpg already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/Precision_Recall_05102018_Papier_non_papier_tres_peu_dense.jpg size_local : 71611 size in s3 : 71611 create time local : 2026-03-04 13:31:58 create time in s3 : 2026-03-04 12:24:16 Precision_Recall_05102018_Papier_non_papier_tres_peu_dense.jpg already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/Result_Summary.txt size_local : 1058 size in s3 : 1058 create time local : 2026-03-04 13:31:58 create time in s3 : 2026-03-04 12:24:17 Result_Summary.txt already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/checkpoint size_local : 99 size in s3 : 99 create time local : 2026-03-04 13:31:58 create time in s3 : 2026-03-04 12:24:17 checkpoint already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/model_checkpoint.ckpt.data-00000-of-00001 size_local : 188538519 size in s3 : 188538519 create time local : 2026-03-04 13:32:01 create time in s3 : 2026-03-04 12:24:27 model_checkpoint.ckpt.data-00000-of-00001 already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/model_checkpoint.ckpt.data-00000-of-00002 size_local : 216572 size in s3 : 216572 create time local : 2026-03-04 13:32:01 create time in s3 : 2026-03-04 12:24:28 model_checkpoint.ckpt.data-00000-of-00002 already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/model_checkpoint.ckpt.data-00001-of-00002 size_local : 32279708 size in s3 : 32279708 create time local : 2026-03-04 13:32:02 create time in s3 : 2026-03-04 12:24:31 model_checkpoint.ckpt.data-00001-of-00002 already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/model_checkpoint.ckpt.index size_local : 28001 size in s3 : 28001 create time local : 2026-03-04 13:32:02 create time in s3 : 2026-03-04 12:24:31 model_checkpoint.ckpt.index already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/model_weights.h5 size_local : 94501976 size in s3 : 94501976 create time local : 2026-03-04 13:32:04 create time in s3 : 2026-03-04 12:24:36 model_weights.h5 already exist and didn't need to update desc size : 2048 Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= module (KerasLayer) (None, 2048) 23561152 _________________________________________________________________ Rungis_amount_dechets_fall_2 (None, 5) 10245 ================================================================= Total params: 23,571,397 Trainable params: 10,245 Non-trainable params: 23,561,152 _________________________________________________________________ Loading Weights... time used to create the model : 7.226611137390137 time used to load_weights : 1.0852603912353516 0it [00:00, ?it/s] 8it [00:00, 78.01it/s] 16it [00:00, 75.51it/s] 24it [00:00, 75.17it/s] 32it [00:00, 74.60it/s] 40it [00:00, 75.34it/s] 48it [00:00, 75.41it/s] 56it [00:00, 75.38it/s] 64it [00:00, 73.27it/s] 73it [00:00, 76.21it/s] 82it [00:01, 78.92it/s] 90it [00:01, 77.98it/s] 98it [00:01, 72.82it/s] 106it [00:01, 72.00it/s]temp/1781113827_2420980_1418250315_504b457388f736f4f87c74c3906c52b5.jpg temp/1781113827_2420980_1418250314_3383d961ef510b3e77855457c940014d.jpg temp/1781113827_2420980_1418250313_f34ab0fddbd920075bb1f2e9d93049ed.jpg temp/1781113827_2420980_1418250311_0e58c2a0d98ff3be807140b8744300d7.jpg temp/1781113827_2420980_1418250309_f9b1b863f999897e0e95cc806d8feb4e.jpg temp/1781113827_2420980_1418250308_e8726a4cd316fa7211bb4fd03d2eb0d8.jpg temp/1781113827_2420980_1418250281_be5895295047b1b11993236c8a836d44.jpg temp/1781113827_2420980_1418250280_aa4ac0722fb3b40b995d293884ab2e81.jpg temp/1781113827_2420980_1418250279_6b446a3c2a4137c3f1402ec1eb9f1c8a.jpg temp/1781113827_2420980_1418250277_d84d3d4a169d61ef07013be63614361f.jpg temp/1781113827_2420980_1418250276_2b5f616647bbd01c4d4b363182875a48.jpg temp/1781113827_2420980_1418250275_cbfa125852bac33604a85f182303607e.jpg temp/1781113827_2420980_1418250262_1fbf76a45063c1b277240afdec681f5e.jpg temp/1781113827_2420980_1418250261_da4941f7e1a776864c040ddd4096bf92.jpg temp/1781113827_2420980_1418250260_2b3e3d645c28147730b11bf21c66694d.jpg temp/1781113827_2420980_1418250259_c1ea00b4f5d559d13896d6a690f0107f.jpg temp/1781113827_2420980_1418250258_ec8ce394e5323afb6784ab6429fecf7c.jpg temp/1781113827_2420980_1418250257_f4b4a636589f7a3d8e5edb728b059793.jpg temp/1781113827_2420980_1418250247_0444eba279e4754dc71a8422e0fc5e49.jpg temp/1781113827_2420980_1418250246_50c113c27dd4382c3d1dc0e5def55941.jpg temp/1781113827_2420980_1418250245_e921e3d92aac25555cb81b130100ce26.jpg temp/1781113827_2420980_1418250244_bc818dd692fdbeb27e35febc781f0e8f.jpg temp/1781113827_2420980_1418250242_6256dff4dc6c2e4f2e5a2902e30ad1b6.jpg temp/1781113827_2420980_1418250241_36c0ce4d7820a75e45296718cc7931c6.jpg temp/1781113827_2420980_1418250227_d4e167b5fbd4fdb5a1d28e1211dd6215.jpg temp/1781113827_2420980_1418250226_18325a5943ca37957f3ca2cf1c77473f.jpg temp/1781113827_2420980_1418250225_a3b774279fac5f1bcbc11f24e40b9ad5.jpg temp/1781113827_2420980_1418250224_1038fcc5aa1eb8c8cd4d1347e46ba32f.jpg temp/1781113827_2420980_1418250223_e2fa80e1e2f918e4d525801698032ca8.jpg temp/1781113827_2420980_1418250222_f247a3ce87bf73eb46e2aedf55f28bc2.jpg temp/1781113827_2420980_1418248930_63734e7b936b40245a3fac882c4087a8.jpg temp/1781113827_2420980_1418248929_4c8a35ab075a46d6c7469e24da9a306f.jpg temp/1781113827_2420980_1418248928_ade063f8f843f7af07ba99bc34dc259e.jpg temp/1781113827_2420980_1418248927_171909317e5f0de45bc1641025bf4c34.jpg temp/1781113827_2420980_1418248926_feedc04ffbc33e1eeab039c748423254.jpg temp/1781113827_2420980_1418248925_b3c075aa79ddceb984f6e521063d453a.jpg temp/1781113827_2420980_1418248862_7d4b7055d43040c2a777eb8b919bc47b.jpg temp/1781113827_2420980_1418248861_0ca5a9a3e2b41ee18e73b259e7f6a9ad.jpg temp/1781113827_2420980_1418248860_290c9afb0bb6b3d41c7377da84d81947.jpg temp/1781113827_2420980_1418248859_4424b33ae6ec0e1a58b3272d59b629d7.jpg temp/1781113827_2420980_1418248858_d3698fcb8be8409e808d4f507da0760e.jpg temp/1781113827_2420980_1418248857_345eb06447226b28dcd1ba53d582ed57.jpg temp/1781113827_2420980_1418248488_a2f601ec4ffc0830976270ac3445dfbe.jpg temp/1781113827_2420980_1418248487_dec2fc8bc770485bfe36fc3b87570cc5.jpg temp/1781113827_2420980_1418248486_99c75a243a325f2ef23b1b7346283f0a.jpg temp/1781113827_2420980_1418248484_770ac2095e22024368983bcb487aed63.jpg temp/1781113827_2420980_1418248483_7e329608ac5039445514104a5cafdd50.jpg temp/1781113827_2420980_1418248481_cda87a7f891d700c4c299a1d07b063ec.jpg temp/1781113827_2420980_1418247548_c10510b2c061bdc9b247391c8fa0faf1.jpg temp/1781113827_2420980_1418247547_dd861bf37ceeb4c4582cae3a2a89681f.jpg temp/1781113827_2420980_1418247546_935374799d9cf0e0e8026d2d8047d015.jpg temp/1781113827_2420980_1418247545_e440089bc7dd759c52069f10ac8072f1.jpg temp/1781113827_2420980_1418247544_abe1317c481cdfc913265afe48da7442.jpg temp/1781113827_2420980_1418247543_0e4b9e2ec54d566ecfa440f48b30a7bc.jpg temp/1781113827_2420980_1418246552_2f97906ded9e010b6e0a9451b8d72325.jpg temp/1781113827_2420980_1418246550_f372a83fe7389ac7b9d1008f466743a2.jpg temp/1781113827_2420980_1418246549_9c8389dc01ee35efaf187a615da695b5.jpg temp/1781113827_2420980_1418246548_5793372bfcdc78dab6b32fbc54706c83.jpg temp/1781113827_2420980_1418246547_b69a11a4a83684ad7eca91f68c1851f2.jpg temp/1781113827_2420980_1418246546_96ce4838f5c389e1189266d419a757e5.jpg temp/1781113827_2420980_1418246533_5a74755eabe743666725e3b0be542266.jpg temp/1781113827_2420980_1418246532_bf8420deb4ecdd8a3fa52787f21e0e64.jpg temp/1781113827_2420980_1418246530_93d54052394b97e7a7a10a933cec2228.jpg temp/1781113827_2420980_1418246529_2699e548ba7b923c751f43c401dd7eb2.jpg temp/1781113827_2420980_1418246528_a89ef5830f7f08dbc9fef3003bd3fc73.jpg temp/1781113827_2420980_1418246527_d72e946bfd4f53d029b2473bba93557f.jpg temp/1781113827_2420980_1418246518_f04cf26da4bb600ca1ae5d4d644e832c.jpg temp/1781113827_2420980_1418246517_6ed34b9cf8ed2129a115f73f45d3b537.jpg temp/1781113827_2420980_1418246515_cf6f832d2b0b0a682d011a9de0d3336a.jpg temp/1781113827_2420980_1418246513_eb86ff1a70ed87ffeff907c61ee3566b.jpg temp/1781113827_2420980_1418246512_e9de886327d4889728e93146f7bc217d.jpg temp/1781113827_2420980_1418246511_4ac20e196e2d082233497c61bd981b83.jpg temp/1781113827_2420980_1418246234_f3cc7e902f22e585b14ef1f5fe3d40e3.jpg temp/1781113827_2420980_1418246233_9c42120da3bb36e1619aafa7d9247939.jpg temp/1781113827_2420980_1418246231_f9227fce133d58ec062d595b16664a10.jpg temp/1781113827_2420980_1418246230_6dad194d04b26981682e95ffd6dd1bb3.jpg temp/1781113827_2420980_1418246229_ca8a067d3b5389c5c625e7298c876fe2.jpg temp/1781113827_2420980_1418246228_87a46d62e67ee57adae7b03a557d44b4.jpg temp/1781113827_2420980_1418246219_2ed0ca7be2f5840ad00e5a11e4300f37.jpg temp/1781113827_2420980_1418246218_36a502bb31f692a0d2409909b126a629.jpg temp/1781113827_2420980_1418246217_df7f5bc9ce064bacb6d19b140839536c.jpg temp/1781113827_2420980_1418246216_781a61f7f0be4241223c4add71c01d26.jpg temp/1781113827_2420980_1418246215_0cc9b4673daf4170721ee8e88c9c69b5.jpg temp/1781113827_2420980_1418246214_cca16bb3ea1cbf980a04297ccaa818bd.jpg temp/1781113827_2420980_1418246204_f26a92d19339f44addd75dd5a853507a.jpg temp/1781113827_2420980_1418246202_79bff636e8f06e00f3ae02e0a9210bb4.jpg temp/1781113827_2420980_1418246201_7780e3594f27bab50598c459a5ed3ac9.jpg temp/1781113827_2420980_1418246200_722800a847ad4070bcad7e785f7ce3ab.jpg temp/1781113827_2420980_1418246199_3d3b661e335d43fb578271491b98c189.jpg temp/1781113827_2420980_1418246198_35c630e66cdfc4fdb25dbfe16b06b76b.jpg temp/1781113827_2420980_1418246019_fecdd3c8ef58c8e925a8d36a77ff2390.jpg temp/1781113827_2420980_1418246018_ef4f11e83286fb5cd73fed594daaae3a.jpg temp/1781113827_2420980_1418246016_2299f27bb930a7bca11e40053a14e44d.jpg temp/1781113827_2420980_1418246014_81759396f45eeb994ed1e835a4005fde.jpg temp/1781113827_2420980_1418245521_63e6336669a215ee60ced72cec130d23.jpg temp/1781113827_2420980_1418245472_60cd7098428bde49cbd58db8ecf970ff.jpg temp/1781113827_2420980_1418244735_ce17d1925704c942771ecdc9dd65da68.jpg temp/1781113827_2420980_1418244734_cc66a8f9b31b4e15d32e46daaf3d86ba.jpg temp/1781113827_2420980_1418244733_a87863e89620b2d921f6113857870202.jpg temp/1781113827_2420980_1418244732_47001b7145c2f5a248631bbfa5064a80.jpg temp/1781113827_2420980_1418244731_4f134084bf0b52b2ee0fd74fa15e2a54.jpg temp/1781113827_2420980_1418244730_7a5da00ea78fdafb0a0637ce1c93d2ee.jpg temp/1781113827_2420980_1418244715_32fc180c6004179baac795bb9c066ad6.jpg temp/1781113827_2420980_1418244714_2b3dd3d392747d6b1589be4d4fe41bb4.jpg temp/1781113827_2420980_1418244713_5aae62345f5ea573137efeccda6d7735.jpg temp/1781113827_2420980_1418244712_4e1f47c5183d7877ae6290feabe5e14f.jpg temp/1781113827_2420980_1418244711_c437b3b1b9a7a9bf3d7447a35b192b8d.jpg temp/1781113827_2420980_1418244710_3ddafa1ceef20893f3844f0e5868aeef.jpg temp/1781113827_2420980_1418244657_252c1852a8b3ba0448060d378bd6f1d1.jpg temp/1781113827_2420980_1418244656_2120cf16c4faa3d3154457d81eb4904b.jpg temp/1781113827_2420980_1418244655_99c37adb309cff1423bfb8d06f68b049.jpg temp/1781113827_2420980_1418244654_4f62d1ed0d0896cadf713af9cf0fa637.jpg temp/1781113827_2420980_1418244653_a053fddfcee4a78223d553adae5b62d2.jpg 114it [00:01, 70.62it/s] 123it [00:01, 73.17it/s] 131it [00:01, 74.74it/s] 139it [00:01, 75.57it/s] 148it [00:01, 77.36it/s] 156it [00:02, 76.92it/s] 164it [00:02, 73.44it/s] 173it [00:02, 75.33it/s] 182it [00:02, 77.54it/s] 190it [00:02, 77.92it/s] 200it [00:02, 76.79it/s]2026-06-10 19:53:14.713235: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2026-06-10 19:53:14.723462: E tensorflow/stream_executor/cuda/cuda_blas.cc:238] failed to create cublas handle: CUBLAS_STATUS_NOT_INITIALIZED 2026-06-10 19:53:14.883041: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2026-06-10 19:53:14.885311: E tensorflow/stream_executor/cuda/cuda_dnn.cc:329] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR 2026-06-10 19:53:14.886666: E tensorflow/stream_executor/cuda/cuda_dnn.cc:329] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR temp/1781113827_2420980_1418244652_c22bb8c2d995f0801090b3f121c3f268.jpg temp/1781113827_2420980_1418244638_a5e258a2b6c41dbf0093c2f3a520248f.jpg temp/1781113827_2420980_1418244636_1805d52b81590514f8ca041f2670e2c6.jpg temp/1781113827_2420980_1418244635_12413d6d9cd16fe2b425433ab5fa0d7a.jpg temp/1781113827_2420980_1418244634_61d7fb35f1f973463dc67618ad445bf1.jpg temp/1781113827_2420980_1418244633_5ac8ad246e673466ec42102e64add1da.jpg temp/1781113827_2420980_1418244632_7a29a028676d0201bf234500f3e45602.jpg temp/1781113827_2420980_1418244623_5974d339441d8c739707e8ee5972e44f.jpg temp/1781113827_2420980_1418244622_22c0f624ebc8eb54aa118502d129a076.jpg temp/1781113827_2420980_1418244619_47d47cbf290e23cf3ab5bcc1a9cf3c8c.jpg temp/1781113827_2420980_1418244584_e5f7fa4890033745cb28e13665101c1a.jpg temp/1781113827_2420980_1418244524_73b2d331a49358e741ebb58c5799a62f.jpg temp/1781113827_2420980_1418244522_e3cd5845e30b06b5e5633c348b8ce77d.jpg temp/1781113827_2420980_1418243537_9821266af03b9b0bebee81d4e505295b.jpg temp/1781113827_2420980_1418243536_431ca0e7a5a65db4b25eec8dfbe34210.jpg temp/1781113827_2420980_1418243535_a32100e21dad43360be089dff71ce49d.jpg temp/1781113827_2420980_1418243534_a52f2fdc63fae3ef22056c8d0658f5f7.jpg temp/1781113827_2420980_1418243533_76816dd8d16ad193c4d703699152dd8f.jpg temp/1781113827_2420980_1418243532_4d351596debc768ba0037848328d1de4.jpg temp/1781113827_2420980_1418243530_18839c1bd14da00fae7299cd5e83b45b.jpg temp/1781113827_2420980_1418243529_9da7f708ff353a22dfff194cc1bc4b74.jpg temp/1781113827_2420980_1418243528_eaabb1eaf617308ece5cef51eed1f635.jpg temp/1781113827_2420980_1418243527_1ab91e041ea4509a563da084c5e0fdfc.jpg temp/1781113827_2420980_1418243526_6451379d906555796ed1ec3b0f287486.jpg temp/1781113827_2420980_1418243525_68c9abf129a53ceea859da64b31d4c55.jpg temp/1781113827_2420980_1418243523_c9fafde631ab926b4f1c2eddba753a7a.jpg temp/1781113827_2420980_1418243522_0f4727b40366f467301b2976ae1a1d44.jpg temp/1781113827_2420980_1418243521_9e7a234f7a7046c259c621bfc5d56d01.jpg temp/1781113827_2420980_1418243520_e955b97bafc71ffc3a44bff9984fc82a.jpg temp/1781113827_2420980_1418243519_ad22592ae80cbad8a08e797095d4b6dc.jpg temp/1781113827_2420980_1418243518_dc29adb216b0afde0e92d38617261d1e.jpg temp/1781113827_2420980_1418243516_4dd0bc446e5a3d43bdb926f654aee598.jpg temp/1781113827_2420980_1418243515_af38303716e9099996adb13f8ca8fb81.jpg temp/1781113827_2420980_1418243514_ee524366340ad742d7385c0163df876d.jpg temp/1781113827_2420980_1418243513_f095a5b96024aa774078685ba589b6c3.jpg temp/1781113827_2420980_1418243512_9e6af2a7e2227e96fe17b242e605ecb9.jpg temp/1781113827_2420980_1418243511_0f621f267e9490971da0358badf549db.jpg temp/1781113827_2420980_1418243503_fe66538cdadbd8a04991301874c440b5.jpg temp/1781113827_2420980_1418243502_9695e9941b0538ce03a1f71b96ce43d2.jpg temp/1781113827_2420980_1418243501_dcfa1cf37246e812f3becfad96898d6b.jpg temp/1781113827_2420980_1418243484_01beec0887d672375c9ac2e952cf7c34.jpg temp/1781113827_2420980_1418243449_1b9464b2b9b9b5b513c8e32bb2bc195a.jpg temp/1781113827_2420980_1418243376_e1f080a78dc2a9dd2f36388eb8d0d164.jpg temp/1781113827_2420980_1418241477_4731bb34b332df8c55b3a3423af0c876.jpg temp/1781113827_2420980_1418241476_ea6c7c5cb2c726005e06a25023551783.jpg temp/1781113827_2420980_1418241475_7c14d4a493b3ff36221aec4396c45a82.jpg temp/1781113827_2420980_1418241474_ae640a25121c67a2256e0bb6af37de7b.jpg temp/1781113827_2420980_1418241473_ca18011181cee8010775f745e3c27c30.jpg temp/1781113827_2420980_1418241472_9010576746e84daa0aef1859f595f7e4.jpg temp/1781113827_2420980_1418239318_1fb63aa528c442619a26040bf2a982d1.jpg temp/1781113827_2420980_1418239317_897ee1990581b9facfc0b23e3d227393.jpg temp/1781113827_2420980_1418239316_48d90437d6454a7a1177f610469adad0.jpg temp/1781113827_2420980_1418239315_8f1e90a127dc8bb2c17aac5f18c807a0.jpg temp/1781113827_2420980_1418239314_6985174f1083d72c11ab4eb4f8a651fe.jpg temp/1781113827_2420980_1418239313_02eeec394e1909726a652792fb759c9a.jpg temp/1781113827_2420980_1418239310_f3f4276c3e6a744106b41028e44d232c.jpg temp/1781113827_2420980_1418239309_d106fc2217eb9ddbb15da4cac180a090.jpg temp/1781113827_2420980_1418239308_1bdb9c2b64cfd4117142e0364a703ef1.jpg temp/1781113827_2420980_1418239307_941069eebc16983829fb4f08437731d3.jpg temp/1781113827_2420980_1418239306_c2bc470bfc0aa4713ed08615937a1740.jpg temp/1781113827_2420980_1418239287_7db69d4daa4ccde5d8dfa1b7c1e519a5.jpg temp/1781113827_2420980_1418239067_c1f69b8852c0430b0b6fabc11d1b881a.jpg temp/1781113827_2420980_1418239066_1a7e481c03de105c1526757a27025047.jpg temp/1781113827_2420980_1418239065_da64ee1fc424e85498b5bc7e75c64d4d.jpg temp/1781113827_2420980_1418239064_d30252fff858dbcfd190db5aa8c8e62f.jpg temp/1781113827_2420980_1418239063_a9a2ac7b632048de55967724be11728a.jpg temp/1781113827_2420980_1418239062_f8d4795537c755d195a1c30e8d87fdf4.jpg temp/1781113827_2420980_1418239061_9d805eb7adaf412b79d39836eb0ea13a.jpg temp/1781113827_2420980_1418239060_636d8456af9c3b1fe5041710a2eca5ac.jpg temp/1781113827_2420980_1418239059_e46e5007afb0087ea70eef0d51ae2660.jpg temp/1781113827_2420980_1418239058_4edaedf1fbe0490e9e855ee18dab7a1d.jpg temp/1781113827_2420980_1418239057_123d14c7b2d3bfa20e3fd853704a6912.jpg temp/1781113827_2420980_1418239056_51530474c31db05fceaf4121037a7188.jpg temp/1781113827_2420980_1418239013_55a0bc8da862726aece822ecea163add.jpg temp/1781113827_2420980_1418239012_c65b741062946e20104b28dcbf164fa5.jpg temp/1781113827_2420980_1418239011_729ad485a59c9c7f1bc582d96dea364b.jpg temp/1781113827_2420980_1418239010_240bb2c31261e84ecf01193ad5352c24.jpg temp/1781113827_2420980_1418239008_ad8abdd7a7ea4f72ffdde5c76f8eca1c.jpg temp/1781113827_2420980_1418239007_2157b8d30298102b9cc1dd312147e8ec.jpg temp/1781113827_2420980_1418239006_782309edd70e9deecf1ccfa133ce8468.jpg temp/1781113827_2420980_1418239005_7b6f42441addfc7f7de6f78121462ccf.jpg temp/1781113827_2420980_1418239004_0a4a75992e26d3a64132a926372eccc8.jpg temp/1781113827_2420980_1418239003_5a002b5238946f224a363aeede5634b9.jpg temp/1781113827_2420980_1418239002_2b2b3af72107e83051275c682b779482.jpg temp/1781113827_2420980_1418239001_54410665bd3659050bff7b323574bab6.jpg temp/1781113827_2420980_1418237840_c456e0a68ba75e9be224be0c196a2a32.jpg temp/1781113827_2420980_1418237839_fcdca1443910b541f8b1f9724addf1cd.jpg Found 200 images belonging to 1 classes. begin to do the prediction : ERROR in datou_step_exec, will save and exit ! Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above. [[{{node model/module/StatefulPartitionedCall/StatefulPartitionedCall/StatefulPartitionedCall/conv1/StatefulPartitionedCall/Conv2D}}]] [Op:__inference_predict_function_34490] Function call stack: predict_function File "/home/admin/workarea/git/Velours/python/mtr/datou/datou_lib.py", line 2340, in datou_exec output = datou_step_exec(sNext, args, cache, context, map_info, verbose, mtr_user_id) File "/home/admin/workarea/git/Velours/python/mtr/datou/datou_lib.py", line 2534, in datou_step_exec return lib_process.datou_step_tfhub2(param, json_param, args, cache, context, map_info, verbose) File "/home/admin/workarea/git/Velours/python/mtr/datou/lib_step_exec/lib_step_process.py", line 3171, in datou_step_tfhub2 classes, outputs, features = this_model.predict_image_paths(list_paths, keep_aspect_ratio=keep_aspect_ratio, File "/home/admin/workarea/git/Velours/python/mtr/tfhub2/evaluate.py", line 288, in predict_image_paths Y_pred, F_pred = self.model.predict(valid_generator, validation_steps) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/training.py", line 88, in _method_wrapper return method(self, *args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/training.py", line 1268, in predict tmp_batch_outputs = predict_function(iterator) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py", line 580, in __call__ result = self._call(*args, **kwds) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py", line 650, in _call return self._concrete_stateful_fn._filtered_call(canon_args, canon_kwds) # pylint: disable=protected-access File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 1661, in _filtered_call return self._call_flat( File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 1745, in _call_flat return self._build_call_outputs(self._inference_function.call( File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 593, in call outputs = execute.execute( File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/execute.py", line 59, in quick_execute tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name, [1418250315, 1418250314, 1418250313, 1418250311, 1418250309, 1418250308, 1418250281, 1418250280, 1418250279, 1418250277, 1418250276, 1418250275, 1418250262, 1418250261, 1418250260, 1418250259, 1418250258, 1418250257, 1418250247, 1418250246, 1418250245, 1418250244, 1418250242, 1418250241, 1418250227, 1418250226, 1418250225, 1418250224, 1418250223, 1418250222, 1418248930, 1418248929, 1418248928, 1418248927, 1418248926, 1418248925, 1418248862, 1418248861, 1418248860, 1418248859, 1418248858, 1418248857, 1418248488, 1418248487, 1418248486, 1418248484, 1418248483, 1418248481, 1418247548, 1418247547, 1418247546, 1418247545, 1418247544, 1418247543, 1418246552, 1418246550, 1418246549, 1418246548, 1418246547, 1418246546, 1418246533, 1418246532, 1418246530, 1418246529, 1418246528, 1418246527, 1418246518, 1418246517, 1418246515, 1418246513, 1418246512, 1418246511, 1418246234, 1418246233, 1418246231, 1418246230, 1418246229, 1418246228, 1418246219, 1418246218, 1418246217, 1418246216, 1418246215, 1418246214, 1418246204, 1418246202, 1418246201, 1418246200, 1418246199, 1418246198, 1418246019, 1418246018, 1418246016, 1418246014, 1418245521, 1418245472, 1418244735, 1418244734, 1418244733, 1418244732, 1418244731, 1418244730, 1418244715, 1418244714, 1418244713, 1418244712, 1418244711, 1418244710, 1418244657, 1418244656, 1418244655, 1418244654, 1418244653, 1418244652, 1418244638, 1418244636, 1418244635, 1418244634, 1418244633, 1418244632, 1418244623, 1418244622, 1418244619, 1418244584, 1418244524, 1418244522, 1418243537, 1418243536, 1418243535, 1418243534, 1418243533, 1418243532, 1418243530, 1418243529, 1418243528, 1418243527, 1418243526, 1418243525, 1418243523, 1418243522, 1418243521, 1418243520, 1418243519, 1418243518, 1418243516, 1418243515, 1418243514, 1418243513, 1418243512, 1418243511, 1418243503, 1418243502, 1418243501, 1418243484, 1418243449, 1418243376, 1418241477, 1418241476, 1418241475, 1418241474, 1418241473, 1418241472, 1418239318, 1418239317, 1418239316, 1418239315, 1418239314, 1418239313, 1418239310, 1418239309, 1418239308, 1418239307, 1418239306, 1418239287, 1418239067, 1418239066, 1418239065, 1418239064, 1418239063, 1418239062, 1418239061, 1418239060, 1418239059, 1418239058, 1418239057, 1418239056, 1418239013, 1418239012, 1418239011, 1418239010, 1418239008, 1418239007, 1418239006, 1418239005, 1418239004, 1418239003, 1418239002, 1418239001, 1418237840, 1418237839] begin to insert list_values into mtr_datou_result : length of list_values in save_final : 200 time used for this insertion : 0.08298635482788086 save_final ERROR in last step tfhub_classification2, Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above. [[{{node model/module/StatefulPartitionedCall/StatefulPartitionedCall/StatefulPartitionedCall/conv1/StatefulPartitionedCall/Conv2D}}]] [Op:__inference_predict_function_34490] Function call stack: predict_function time spend for datou_step_exec : 121.28107523918152 time spend to save output : 0.0927577018737793 total time spend for step 0 : 121.3738329410553 need to delete datou_research and reload, so keep current state 1 caffe_path_current : About to save ! 2 After save, about to update current ! 14.06user 10.12system 2:51.36elapsed 14%CPU (0avgtext+0avgdata 1742920maxresident)k 2649296inputs+5453432outputs (129major+670491minor)pagefaults 0swaps