python /home/admin/mtr/script_for_cron.py -j default -m 20 -a 'python3 ~/workarea/git/Velours/python/prod/datou.py -j batch_current -C 2562432' -s traitement_4892 -M 0 -S 0 -U 100,80,95 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/home/admin/workarea/git/apy', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3224038 load datou : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : step 0 init_dummy_multi_datou is not linked in the step_by_step architecture ! WARNING : step 1294 init_dummy_multi_datou is not linked in the step_by_step architecture ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : (photo_id, hashtag_id, score_max) was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec no input labels no input values updating current state to 1 we have a portfolio with more photos than limit : 7456>1000 please execute split_portfolio.py -i 20244820 -l 1000 size over we load limit photo not treated list_input_json: {} Current got : datou_id : 4892, datou_cur_ids : ['2562432'] with mtr_portfolio_ids : ['20244820'] and first list_photo_ids : [] new path : /proc/3224038/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! DataTypes for each output/input checked ! List Step Type Loaded in datou : tfhub_classification2, argmax over limit max, limiting to limit_max 300 list_input_json : {} origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 300 ; length of list_pids : 300 ; length of list_args : 300 time to download the photos : 65.88580656051636 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 2 step1:tfhub_classification2 Fri Feb 7 11:30:12 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou_step TFHub with tf2 ! nombre de thcls : 2 we are using the classfication for multi_thcl [3513, 3890] begin to check gpu status inside check gpu memory 2025-02-07 11:30:15.735891: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-07 11:30:15.744787: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-07 11:30:15.745125: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-07 11:30:15.747246: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-07 11:30:15.749274: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-07 11:30:15.749583: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-07 11:30:15.751700: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-07 11:30:15.752687: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-07 11:30:15.756932: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-07 11:30:15.757997: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-07 11:30:15.758488: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-07 11:30:15.783316: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-07 11:30:15.784995: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fe640000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-07 11:30:15.785061: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-07 11:30:16.003659: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x36b23460 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-07 11:30:16.003706: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-07 11:30:16.004244: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-07 11:30:16.004330: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-07 11:30:16.004347: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-07 11:30:16.004363: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-07 11:30:16.004377: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-07 11:30:16.004392: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-07 11:30:16.004406: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-07 11:30:16.004421: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-07 11:30:16.005225: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-07 11:30:16.005269: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-07 11:30:16.005783: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-07 11:30:16.005796: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-07 11:30:16.005804: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-07 11:30:16.006653: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3096 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) l 3637 free memory gpu now : 3248 max_wait_temp : 1 max_wait : 5 1 Physical GPUs, 1 Logical GPUs tagging for thcl : 3513 To do loadFromThcl(), then load ParamDescType : thcl3513 thcls : [{'id': 3513, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2_tf', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 4557, 'photo_desc_type': 5767, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] thcl {'id': 3513, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2_tf', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 4557, 'photo_desc_type': 5767, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'} Update svm_hashtag_type_desc : 5767 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5767, 'Rungis_amount_dechets_fall_2018_v2_tf', 2048, 2048, 'Rungis_amount_dechets_fall_2018_v2_tf', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 3, 16, 15, 52, 10), datetime.datetime(2023, 3, 16, 15, 52, 10)) model_name : Rungis_amount_dechets_fall_2018_v2_tf model_param file didn't exist model_name : Rungis_amount_dechets_fall_2018_v2_tf model_type : tf_classification2 list file need : ['Confusion_Matrix.png', 'Precision_Recall_05102018_Papier_non_papier_dense.jpg', 'Precision_Recall_05102018_Papier_non_papier_peu_dense.jpg', 'Precision_Recall_05102018_Papier_non_papier_presque_vide.jpg', 'Precision_Recall_05102018_Papier_non_papier_tres_dense.jpg', 'Precision_Recall_05102018_Papier_non_papier_tres_peu_dense.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00001', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file exist in s3 : ['Confusion_Matrix.png', 'Precision_Recall_05102018_Papier_non_papier_dense.jpg', 'Precision_Recall_05102018_Papier_non_papier_peu_dense.jpg', 'Precision_Recall_05102018_Papier_non_papier_presque_vide.jpg', 'Precision_Recall_05102018_Papier_non_papier_tres_dense.jpg', 'Precision_Recall_05102018_Papier_non_papier_tres_peu_dense.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00001', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file manque in s3 : [] 2025-02-07 11:30:22.637142: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.02G (3246391296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/Confusion_Matrix.png size_local : 67810 size in s3 : 67810 create time local : 2023-10-30 16:21:37 create time in s3 : 2023-10-30 14:09:29 Confusion_Matrix.png already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/Precision_Recall_05102018_Papier_non_papier_dense.jpg size_local : 73949 size in s3 : 73949 create time local : 2023-10-30 16:21:38 create time in s3 : 2023-10-30 14:09:30 Precision_Recall_05102018_Papier_non_papier_dense.jpg already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/Precision_Recall_05102018_Papier_non_papier_peu_dense.jpg size_local : 85572 size in s3 : 85572 create time local : 2023-10-30 16:21:38 create time in s3 : 2023-10-30 14:09:39 Precision_Recall_05102018_Papier_non_papier_peu_dense.jpg already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/Precision_Recall_05102018_Papier_non_papier_presque_vide.jpg size_local : 72361 size in s3 : 72361 create time local : 2023-10-30 16:21:38 create time in s3 : 2023-10-30 14:09:37 Precision_Recall_05102018_Papier_non_papier_presque_vide.jpg already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/Precision_Recall_05102018_Papier_non_papier_tres_dense.jpg size_local : 83567 size in s3 : 83567 create time local : 2023-10-30 16:21:38 create time in s3 : 2023-10-30 14:09:48 Precision_Recall_05102018_Papier_non_papier_tres_dense.jpg already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/Precision_Recall_05102018_Papier_non_papier_tres_peu_dense.jpg size_local : 71611 size in s3 : 71611 create time local : 2023-10-30 16:21:38 create time in s3 : 2023-10-30 14:09:29 Precision_Recall_05102018_Papier_non_papier_tres_peu_dense.jpg already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/Result_Summary.txt size_local : 1058 size in s3 : 1058 create time local : 2023-10-30 16:21:38 create time in s3 : 2023-10-30 14:09:30 Result_Summary.txt already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/checkpoint size_local : 99 size in s3 : 99 create time local : 2023-10-30 16:21:38 create time in s3 : 2023-10-30 14:09:36 checkpoint already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/model_checkpoint.ckpt.data-00000-of-00001 size_local : 188538519 size in s3 : 188538519 create time local : 2023-10-30 16:21:41 create time in s3 : 2023-10-30 14:09:40 model_checkpoint.ckpt.data-00000-of-00001 already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/model_checkpoint.ckpt.data-00000-of-00002 size_local : 216572 size in s3 : 216572 create time local : 2023-10-30 16:21:41 create time in s3 : 2023-03-16 14:52:09 model_checkpoint.ckpt.data-00000-of-00002 already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/model_checkpoint.ckpt.data-00001-of-00002 size_local : 32279708 size in s3 : 32279708 create time local : 2023-10-30 16:21:42 create time in s3 : 2023-03-16 14:52:07 model_checkpoint.ckpt.data-00001-of-00002 already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/model_checkpoint.ckpt.index size_local : 28001 size in s3 : 28001 create time local : 2023-10-30 16:21:42 create time in s3 : 2023-10-30 14:09:30 model_checkpoint.ckpt.index already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/model_weights.h5 size_local : 94501976 size in s3 : 94501976 create time local : 2023-10-30 16:21:44 create time in s3 : 2023-10-30 14:09:37 model_weights.h5 already exist and didn't need to update desc size : 2048 Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= module (KerasLayer) (None, 2048) 23561152 _________________________________________________________________ Rungis_amount_dechets_fall_2 (None, 5) 10245 ================================================================= Total params: 23,571,397 Trainable params: 10,245 Non-trainable params: 23,561,152 _________________________________________________________________ Loading Weights... time used to create the model : 8.065385580062866 time used to load_weights : 0.2509734630584717 0it [00:00, ?it/s] 26it [00:00, 254.62it/s] 52it [00:00, 252.82it/s] 83it [00:00, 278.22it/s] 112it [00:00, 282.27it/s]temp/1738924147_3224038_1335031573_c8931dcfdd7bb1d3c78f08b510f88ae0.jpg temp/1738924147_3224038_1335031571_919578aeedcd4e4688d1cd9c5ed696b6.jpg temp/1738924147_3224038_1335031569_5bbc7f3b8bf204fe926f495479049229.jpg temp/1738924147_3224038_1335031568_f47cd93a3024509eabc7cbe06a2ebb81.jpg temp/1738924147_3224038_1335031567_86c7c5fbd465ade1e1f2c9f53bc5042e.jpg temp/1738924147_3224038_1335031566_f143cb5a336a95a7e2eb5e923a553177.jpg temp/1738924147_3224038_1335031562_37a8df34dea80a8c57c6fd9abede805c.jpg temp/1738924147_3224038_1335031561_2feb7eb067ed5ab21b131860a62f9bd5.jpg temp/1738924147_3224038_1335031558_0bb3a61ee26dda2bdca204bac6d616c5.jpg temp/1738924147_3224038_1335031555_f65fd4b5fc5e955c006190883ffc2562.jpg temp/1738924147_3224038_1335031550_b822186478fa0bb4599ee4ad3074bff2.jpg temp/1738924147_3224038_1335031476_fc1b5f7ab16a6cd7a46a184dc197c858.jpg temp/1738924147_3224038_1335031400_665e49936f539e11069a4d27c7fe1618.jpg temp/1738924147_3224038_1335031397_ea7c5097def2e4ca20dc97c5e226130d.jpg temp/1738924147_3224038_1335031394_ec1b7845d506cec16f93c3611c18f9dc.jpg temp/1738924147_3224038_1335031393_21e5d0eae750b372170e673da0d3a769.jpg temp/1738924147_3224038_1335031389_5fc2ecceca95b1b39cf5fad537307f07.jpg temp/1738924147_3224038_1335031386_b3da894ff7c2439a21aff772e9aa6c8c.jpg temp/1738924147_3224038_1335031366_31e9a21ea1fe72e2ab7ec100e43f136c.jpg temp/1738924147_3224038_1335031360_bc959a9d16359615a924899d61468aa7.jpg temp/1738924147_3224038_1335031356_22ec545fb001a0b74c921b786c2dc7f2.jpg temp/1738924147_3224038_1335031344_83c6a2c359f15528b66b6f90db27fc01.jpg temp/1738924147_3224038_1335031343_43f7d39426fc346ff1e7379f4987830b.jpg temp/1738924147_3224038_1335031341_21680e55759dd1075820a34a8468b5a3.jpg temp/1738924147_3224038_1335031328_e44d4c815200fdbd77542c8850aa4fd3.jpg temp/1738924147_3224038_1335031324_c25d3d958ae9f0f2e99026e016e775aa.jpg temp/1738924147_3224038_1335031319_45e84e79bda848fdf7e2a88e1bd1fc2a.jpg temp/1738924147_3224038_1335031315_df4d5ff31a8dc3d88e7be1f454b4163a.jpg temp/1738924147_3224038_1335031312_8b91cba47224ff1382104a4d14937d4f.jpg temp/1738924147_3224038_1335031310_46ff5298bde3896a6015c95256e6a85c.jpg temp/1738924147_3224038_1335031293_c39d321a2524690b1021205246a8e085.jpg temp/1738924147_3224038_1335031292_ea12333905c4c6922edde547020ec016.jpg temp/1738924147_3224038_1335031290_ebd8f22e054e028a782c74f7aff236e9.jpg temp/1738924147_3224038_1335031287_86e5335b8cc1cbdbc8940487a65625f2.jpg temp/1738924147_3224038_1335031286_d015dd555200cc984bcc49e8f2c6e862.jpg temp/1738924147_3224038_1335031284_644f02566880da9a064c02d254515649.jpg temp/1738924147_3224038_1335031267_2205581156e8f6d7d0cb5a72a5f3b4da.jpg temp/1738924147_3224038_1335031264_57b7c22ac0e19ba9963a22cca3c5c047.jpg temp/1738924147_3224038_1335031262_1b60306ce67e3ac45433a4b0f050100d.jpg temp/1738924147_3224038_1335031259_36a463038ca8f3d5435741a9442be8ff.jpg temp/1738924147_3224038_1335031256_9d727d311a52d22404af94e6b3520f43.jpg temp/1738924147_3224038_1335031254_e30eaecd4eaae5056de93a773d23db87.jpg temp/1738924147_3224038_1335031235_560a8fe8ebe98f2bf24c538542e41856.jpg temp/1738924147_3224038_1335031233_b5047a720ad0c34445c2896392cfc8bc.jpg temp/1738924147_3224038_1335031230_9aedff0d478c561540128ba368f10e87.jpg temp/1738924147_3224038_1335031225_0526fa4605fabaf85a9d851ab60c3bc4.jpg temp/1738924147_3224038_1335031221_46cced21611ed18955d10c0e172c6233.jpg temp/1738924147_3224038_1335031220_dd5ba881d2dc83ad84a73746a4ed2375.jpg temp/1738924147_3224038_1335031049_76281a3c1291ecf3a01879e72254346b.jpg temp/1738924147_3224038_1335030946_0ba9862cb575748b93321dcd9bf16dcf.jpg temp/1738924147_3224038_1335030826_93c8b7a30be4987cee0aad29e1938142.jpg temp/1738924147_3224038_1335030821_d310a359c14b9182918645c479256383.jpg temp/1738924147_3224038_1335030814_39f39348e55aa37031d1effb8fe0af3a.jpg temp/1738924147_3224038_1335030787_a10f4555a2079ac88f0cdd9d2ce34774.jpg temp/1738924147_3224038_1335030762_39d8e86581e6316bedbb9a31a54cb273.jpg temp/1738924147_3224038_1335030758_bdbdcf3af6f64659ca01b381d3453888.jpg temp/1738924147_3224038_1335030752_4023de42ef5d9c7f37ec086532b4c2eb.jpg temp/1738924147_3224038_1335030724_03bcfc827d4f252c8e8d82a2d979ca73.jpg temp/1738924147_3224038_1335030719_9e5db1c3b5d8a1bc2ad2d61bbfc028ad.jpg temp/1738924147_3224038_1335030716_fdfa5ea42e573b0bba1b68f0d10d3499.jpg temp/1738924147_3224038_1335030674_ce8a1867581ed2a5d13ad17c6effd9d2.jpg temp/1738924147_3224038_1335030617_32a2ff9dddc7eb0403e3326c54d462b1.jpg temp/1738924147_3224038_1335030614_1552409c43201066a730d6d26153f091.jpg temp/1738924147_3224038_1335030611_a53603e62e818ce5df5f730b9dfa9a4f.jpg temp/1738924147_3224038_1335030609_b7ed13a7aa40d97eca9ac5322d832afb.jpg temp/1738924147_3224038_1335030608_dc249a9f8794564b6704496fa1c7acd4.jpg temp/1738924147_3224038_1335030590_02483fa9c3befecf8799dc36c2ad5d0c.jpg temp/1738924147_3224038_1335030587_293d79942f790086229fd3a4af67802f.jpg temp/1738924147_3224038_1335030578_964bf83ccd30310d15f5b4d2e490729f.jpg temp/1738924147_3224038_1335030574_82b306b8d6c10f80bdb65c2f66f54ed2.jpg temp/1738924147_3224038_1335030569_aea370699ee1aa0788ffc7e29fa527e1.jpg temp/1738924147_3224038_1335030565_cd04c0ad07bc5678a106d310322e77b0.jpg temp/1738924147_3224038_1335030483_e22cf6e7b92e3d0a746663b0d6e2b4c1.jpg temp/1738924147_3224038_1335030479_e11ec5523adf126a42b2136a346dab74.jpg temp/1738924147_3224038_1335030474_474899b53b02811d432ffe89353f6bcf.jpg temp/1738924147_3224038_1335030465_99471a2fcd799df2859d0288b30fb089.jpg temp/1738924147_3224038_1335030457_8a8ad11530d8aa096bf23c557f7aa596.jpg temp/1738924147_3224038_1335030455_f0d11db74ed65a2323634922183d8498.jpg temp/1738924147_3224038_1335030434_e9853d3a3c101d64a91f88ba67694d4a.jpg temp/1738924147_3224038_1335030431_3064d441afe1353de43bd08cce1c5820.jpg temp/1738924147_3224038_1335030373_4b76bf0b9c2dd11b8304abb4878c77f1.jpg temp/1738924147_3224038_1335030342_24066a17ae818890fdd0c6187954fe51.jpg temp/1738924147_3224038_1335030261_40bda266aa85cda538d417c9d2266134.jpg temp/1738924147_3224038_1335030251_d73318dd8f9caa181c2702fa8908c98c.jpg temp/1738924147_3224038_1335030226_d3ccd34f89b1aec63adde6f399e385dd.jpg temp/1738924147_3224038_1335030223_4b87aaa2ef225cbdf10044abcc1a5ba1.jpg temp/1738924147_3224038_1335030222_3fa752638ed3a64e9797ea43981f7a25.jpg temp/1738924147_3224038_1335030221_2f227bdd3a5618a0e2af8e319bce0935.jpg temp/1738924147_3224038_1335030219_b4db8d5f9a01c32d2e2e7e58cf1e845c.jpg temp/1738924147_3224038_1335030216_c28efa1ba220369007da3f9f019d7cf8.jpg temp/1738924147_3224038_1335030139_c264ce9a5437c8f185689306b6d104a6.jpg temp/1738924147_3224038_1335030135_d0a0aecd72335c3bd5e5f4b03c5364f1.jpg temp/1738924147_3224038_1335030132_d0d8f0164df713404ce77d8ff3688bcf.jpg temp/1738924147_3224038_1335030131_3e9bfea37d5255736a29f86fa0710f9d.jpg temp/1738924147_3224038_1335030128_66c8580752ffaf66dd03148495e32946.jpg temp/1738924147_3224038_1335030126_c8cc6eb9c19a678fab0296bbec5ee733.jpg temp/1738924147_3224038_1335030103_301720586e5e58b555a16b1bc7b669da.jpg temp/1738924147_3224038_1335030102_290a6beeda173747c61d14b5a281db06.jpg temp/1738924147_3224038_1335030101_d45ad6275f9ba46a2000e37ca0906c7d.jpg temp/1738924147_3224038_1335030094_96e13f238c2befd90dff623d490db427.jpg temp/1738924147_3224038_1335030091_82666db2487fc1657aabc77831fc4eb2.jpg temp/1738924147_3224038_1335030088_b01a39ed24719ea0891b54d05aafd3e4.jpg temp/1738924147_3224038_1335029918_bde5fbbbff973359893ece199b20042b.jpg temp/1738924147_3224038_1335029916_0231dca5558c07e92d56622d86ccd482.jpg temp/1738924147_3224038_1335029915_1145e9a67c35ace3d49821377af99fac.jpg temp/1738924147_3224038_1335029912_2399429c0e921f8d6cb8279fbd64f42b.jpg temp/1738924147_3224038_1335029908_29806b449699c1dec32e36c35d11e82c.jpg temp/1738924147_3224038_1335029905_e475706b48bca268df987f9417f4fa8e.jpg temp/1738924147_3224038_1335029889_3f95bf18fd9af628427e18bb5f2e1bd2.jpg temp/1738924147_3224038_1335029887_d0a70a5b8a467df6483e4a89a7cd84be.jpg temp/1738924147_3224038_1335029886_a057b027bd51c381e471096782bdada7.jpg temp/1738924147_3224038_1335029884_5e83bba1eed40b8aaddbf790210c2710.jpg temp/1738924147_3224038_1335029881_86e2e07f05364ca865c7221839b955ef.jpg 211it [00:00, 535.47it/s]temp/1738924147_3224038_1335029878_4839ea3cba59c9152b7f0526699d087c.jpg temp/1738924147_3224038_1335029834_8cd033bf411719ebe99f80eb0ef92fce.jpg temp/1738924147_3224038_1335029801_ca7843539ff1af7c9a96b7a667dfd5f4.jpg temp/1738924147_3224038_1335029742_c1af37194bb4b9210f9a40b9f64fb3f8.jpg temp/1738924147_3224038_1335029721_26206dabeac08b09eae7864bfa70195e.jpg temp/1738924147_3224038_1335029703_91057c50768e3700c3d1b41758cf642f.jpg temp/1738924147_3224038_1335029679_ffe0a977afe7d5774976f15d9dcd88a2.jpg temp/1738924147_3224038_1335029642_fa15edab6337f1713179c8a7dee69958.jpg temp/1738924147_3224038_1335029608_43622e97da7fc23ac2263f1361e003a5.jpg temp/1738924147_3224038_1335029568_6c3353c308bf1c474dbb1b12a4edfb16.jpg temp/1738924147_3224038_1335029534_2d964827638214b7a5e99eff1a8c5f47.jpg temp/1738924147_3224038_1335029462_8eaded8dcdfcda202a40cef760ea3936.jpg temp/1738924147_3224038_1335029416_6b6b3550766faf404d684e4c81d43dec.jpg temp/1738924147_3224038_1335029312_ca8c2514f90eb0757bc2814e70d713f0.jpg temp/1738924147_3224038_1335029311_645ef8b4709a63e2c59e66a198e78faa.jpg temp/1738924147_3224038_1335029310_773851555951bc2d742d219fb0b221e6.jpg temp/1738924147_3224038_1335029309_8eb639719de2841e1b7a072feb575427.jpg temp/1738924147_3224038_1335029306_daf58475890c51b30d31f30f4c04d503.jpg temp/1738924147_3224038_1335029303_17480bdf71fbc993364a742886c924b0.jpg temp/1738924147_3224038_1335029297_e84ff33e7a68fd4e64a4c8f85217f117.jpg temp/1738924147_3224038_1335029296_9d8e67188b5da420a8387e324abbec09.jpg temp/1738924147_3224038_1335029294_471f716e26df8a9a39ff04ca82c9f3ae.jpg temp/1738924147_3224038_1335029292_89946f9a29e77708534aea95cc5c7f8b.jpg temp/1738924147_3224038_1335029289_95b390aaf56c3301ffa089e7ade89869.jpg temp/1738924147_3224038_1335029288_ab220ae372d3c352730bf8fab03beb6d.jpg temp/1738924147_3224038_1335029285_ae825843116dded839125df94be9679c.jpg temp/1738924147_3224038_1335029284_49da25284e3ea2b07017ea8f19b8e5a2.jpg temp/1738924147_3224038_1335029281_785dd3c5af49552d6e0c1ec61d968adf.jpg temp/1738924147_3224038_1335029280_7b673573d5fae7c688caba6c0980dfef.jpg temp/1738924147_3224038_1335029277_6011ddc5a2593e509377fdec6816f5d1.jpg temp/1738924147_3224038_1335029275_ffcb3e6a55b79fb7d0fb89471815a608.jpg temp/1738924147_3224038_1335029196_a1fdf588403792dd4c81b74fc8208312.jpg temp/1738924147_3224038_1335029193_03c19c4bcf4320ee441d2a32ed9fdda5.jpg temp/1738924147_3224038_1335029192_5e8547d66a0a865ed9b7eb0026543931.jpg temp/1738924147_3224038_1335029188_dcecc0c60e10bf49c0779a6219d16689.jpg temp/1738924147_3224038_1335029187_a27bbfb4b046222dc50a307ccada943e.jpg temp/1738924147_3224038_1335029183_0de0f7ec2ba8e2b8c4b97844acc6416a.jpg temp/1738924147_3224038_1335029182_b3dda170fc0a8305d2a4e170f8108104.jpg temp/1738924147_3224038_1335029177_b21854eef4fb125ffcca387d6ed25452.jpg temp/1738924147_3224038_1335029176_3cb1922e6f940495743000098e154c2b.jpg temp/1738924147_3224038_1335029172_d11615d83f3cdac5c8f2a558994cb22e.jpg temp/1738924147_3224038_1335029170_6b77582a858da3b0eb031ef105bc5c96.jpg temp/1738924147_3224038_1335029165_8aea03b8fd1478baddcca770d53a0df3.jpg temp/1738924147_3224038_1335029145_3cf5cfebeb7b2a1bbff27aad0f61ea77.jpg temp/1738924147_3224038_1335029142_6d60a2a43805e0e6d22efeb5748b1298.jpg temp/1738924147_3224038_1335029139_d95d467c1474a7fa9ebf7a8bf0d0a243.jpg temp/1738924147_3224038_1335029137_3bc35c147e63184852e0ccf505ed7a87.jpg temp/1738924147_3224038_1335029134_2d1092e92a806295c8f1eb74a3d8c43a.jpg temp/1738924147_3224038_1335029132_f4ee7b11f525061f6ae95fbb707d3561.jpg temp/1738924147_3224038_1335029121_08b430dd198e2b601b19c26136668058.jpg temp/1738924147_3224038_1335029119_bdd06b3883286f455144d69f85c5b380.jpg temp/1738924147_3224038_1335029116_208d352f000cecb636dc01c4322f45bd.jpg temp/1738924147_3224038_1335029113_e1f601d40bebf749bdc8fb20aa4eccee.jpg temp/1738924147_3224038_1335029110_e189fc662ff4e7c991f6e1f974f40ae8.jpg temp/1738924147_3224038_1335029107_3d56e80d0f26ece871ff0f8172e1e2b4.jpg temp/1738924147_3224038_1335029092_45932cf8d86dd2aa714544d9847e5022.jpg temp/1738924147_3224038_1335029045_afc6fb0258c26cd9bf30e8d22ac851e3.jpg temp/1738924147_3224038_1335029040_8cbfda2745360676462f4a6836b61f36.jpg temp/1738924147_3224038_1335029037_93d685c1ad32b362970b2fe6fdad80a1.jpg temp/1738924147_3224038_1335029030_1ce7634c109411d1adb2992298a96fa5.jpg temp/1738924147_3224038_1335029000_a7986655a687ab2e40d4b23610b78850.jpg temp/1738924147_3224038_1335028953_aa0a8732a534408b162a5efd344f426f.jpg temp/1738924147_3224038_1335028951_4f51fa5936a336dbea8ff029ac2abea2.jpg temp/1738924147_3224038_1335028948_e95269acb975c418c1d3d9099b20a267.jpg temp/1738924147_3224038_1335028945_9031713415a2c3df50098c0f94524cd3.jpg temp/1738924147_3224038_1335028942_0a519d61d19dad896151491384809e23.jpg temp/1738924147_3224038_1335028939_64661ba30232a06121bfa1c56a67368e.jpg temp/1738924147_3224038_1335028913_3de29dbc70b3a12b50d7dc6eb58cdfe7.jpg temp/1738924147_3224038_1335028907_aace6076fc84dac3229b2066b02d1ae4.jpg temp/1738924147_3224038_1335028906_bb33209ca3b6cf0149aaed2646c88a26.jpg temp/1738924147_3224038_1335028902_69aa1f83bbb26886e9ed4fda1b38d304.jpg temp/1738924147_3224038_1335028900_bab4a7c4cceeb7e7ac934129ba1aae1e.jpg temp/1738924147_3224038_1335028890_899f2181513aaa29cb88ab58bf8acaa2.jpg temp/1738924147_3224038_1335028888_f421d6f6e2e1cbf1511f07a754c6b4c4.jpg temp/1738924147_3224038_1335028884_fb2673dc8212e43493d2137112c54dc6.jpg temp/1738924147_3224038_1335028873_1ea5554550c6d03f8e518c656be26ed6.jpg temp/1738924147_3224038_1335028871_886e9aaae1d795cdbe2ab33b9aa02531.jpg temp/1738924147_3224038_1335028869_5808453b323d57c8e2b3e575394b20c1.jpg temp/1738924147_3224038_1335028868_11f20cc0a0e10410e650830a84e4e71e.jpg temp/1738924147_3224038_1335028839_ff6626dc234fc2d7eb8d68a057f3ec78.jpg temp/1738924147_3224038_1335028837_46a8245608739c074e29a6e90c4a8ada.jpg temp/1738924147_3224038_1335028834_7198153da63637e4c560494d4737f4c2.jpg temp/1738924147_3224038_1335028832_4e6c746fd09601efd9295f9a05b11494.jpg temp/1738924147_3224038_1335028829_274e44684d018621f553a8a4e6371ea8.jpg temp/1738924147_3224038_1335028828_8da1e22b529e6c1a1830b4bbd380c71e.jpg temp/1738924147_3224038_1335028826_8a6b1d664b39c97bbf84be420bcb5741.jpg temp/1738924147_3224038_1335028825_e4065ad476c4a84999d6c37f469cde13.jpg temp/1738924147_3224038_1335028823_632ab94fd8d5c7132d5296fdb6577a85.jpg temp/1738924147_3224038_1335028821_82ae49c6f912b1e0e3a3ec3d6eb30e67.jpg temp/1738924147_3224038_1335028817_5b06c8b1d95bd9ae21702361423556a7.jpg temp/1738924147_3224038_1335028815_ca0678a1aba1e09018b310580f5526a6.jpg temp/1738924147_3224038_1335028792_7981bca7f50ede26359dfee5a7ba58af.jpg temp/1738924147_3224038_1335028788_0fc8dc05a721acd16998c4bf63db0944.jpg temp/1738924147_3224038_1335028784_18688787b2b16b3b90b84170239f6c11.jpg temp/1738924147_3224038_1335028780_91137c4a142db952d97d3508607eccfd.jpg temp/1738924147_3224038_1335028776_b5333e487c7344a4ceedab27a07ad34d.jpg temp/1738924147_3224038_1335028773_bfbf7bcf2ec743cf7e3034cb05368c42.jpg temp/1738924147_3224038_1335028762_0d93f44ed9a83300d5ce84ee6d993f87.jpg temp/1738924147_3224038_1335028761_fd0a4745595d3ab198e088930c5e1461.jpg temp/1738924147_3224038_1335028760_c3df7caaf178c4e6dbe3d7736127bbb1.jpg temp/1738924147_3224038_1335028759_fea957a32d56240b96a3e73cde8ddf27.jpg temp/1738924147_3224038_1335028756_c03c860d1e6d18768e57bf4f042aa767.jpg temp/1738924147_3224038_1335028753_004b8fd86d5274c3bfbd6a73c6fd63c7.jpg temp/1738924147_3224038_1335028740_7e9049c923b15cfe421ff9a685325f33.jpg temp/1738924147_3224038_1335028736_92931461fb170cc3a10798fd99ddf792.jpg temp/1738924147_3224038_1335028732_e13ed0aa86fed7e652e3bcf878e4b0ad.jpg temp/1738924147_3224038_1335028729_760aacfbe7955834dd1ce637f84f2dd7.jpg temp/1738924147_3224038_1335028726_80659bed0ae1d9a4ede090e4158a1d0e.jpg temp/1738924147_3224038_1335028721_d4db4cf886feff1cc6d795e2d3bcbfc2.jpg temp/1738924147_3224038_1335028705_8a75f4b8daccf8caab94e4bd29547cc5.jpg temp/1738924147_3224038_1335028703_7d8fa08d6cffc5407e357cf583bb8617.jpg temp/1738924147_3224038_1335028701_4b570de2f06b156b437cad30070954fa.jpg temp/1738924147_3224038_1335028700_8a0666a9d7c51ab4abdd9c752bbf582c.jpg 300it [00:00, 524.11it/s]2025-02-07 11:30:27.804765: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-07 11:30:28.167823: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-07 11:30:28.544662: E tensorflow/stream_executor/cuda/cuda_dnn.cc:329] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR 2025-02-07 11:30:28.550865: E tensorflow/stream_executor/cuda/cuda_dnn.cc:329] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR temp/1738924147_3224038_1335028681_0b78e820981b78f3b88fb70a529c70b0.jpg temp/1738924147_3224038_1335028680_cb1f973afd9ea33ec4c583cc9e2223dd.jpg temp/1738924147_3224038_1335028677_68ea3cdec46e777ba2b9f576b7f0f6a9.jpg temp/1738924147_3224038_1335028676_8ea662fbfec959b2d0f2ba02ecd97f2c.jpg temp/1738924147_3224038_1335028673_24f9ea233e25f49e37c425716c7b21aa.jpg temp/1738924147_3224038_1335028672_4fe207231eb5b538d60795300c582dc7.jpg temp/1738924147_3224038_1335028670_fde6b3bb2bd89da9bbcfd213a5c7d359.jpg temp/1738924147_3224038_1335028669_58913d623fe035c864023042bf195664.jpg temp/1738924147_3224038_1335028656_2c4d421d47ef7baaa3c1793d2a50300b.jpg temp/1738924147_3224038_1335028655_a00d878ff2821cab825caa87b32ecfe5.jpg temp/1738924147_3224038_1335028652_0404a9fa3891740e62cad9d437f1ebcb.jpg temp/1738924147_3224038_1335028650_a6cfe94a522d047dc016a405e9330d51.jpg temp/1738924147_3224038_1335028647_ce5bc94f81369a742d0ddabc32b0fc81.jpg temp/1738924147_3224038_1335028646_be1be3f1720f1b85c3016ea3f1a58b3d.jpg temp/1738924147_3224038_1335028644_cfa2138e950818f77a064a1255e6a4fe.jpg temp/1738924147_3224038_1335028643_40e28e4a9ae2e40f55bfd5373f9307e5.jpg temp/1738924147_3224038_1335028616_a5e132e466e23f5999cedd6b4f9aae40.jpg temp/1738924147_3224038_1335028615_1cf625a846cf4379749e63858049dd28.jpg temp/1738924147_3224038_1335028613_cc0a2b6ed30709b8aad87da69811d88d.jpg temp/1738924147_3224038_1335028611_546074cb2c4092a93c590eddfed85967.jpg temp/1738924147_3224038_1335028597_a51752e878f95e37c32dc984b9d12281.jpg temp/1738924147_3224038_1335028594_e74959abcf824b5f2baceef462df6e15.jpg temp/1738924147_3224038_1335028590_2d3be67864749d5c9f24f00d500fd74f.jpg temp/1738924147_3224038_1335028586_bf493b88273cf6da9d5ef7528e83a8bd.jpg temp/1738924147_3224038_1335028576_5200a84b3568d0bea05404e309ed3cc9.jpg temp/1738924147_3224038_1335028570_bfcdd9af22c058d27b8625b652eb1358.jpg temp/1738924147_3224038_1335028542_817c78a84d1cb90b291f434f01f5e351.jpg temp/1738924147_3224038_1335028539_d848bed118cf1536514660abf4f66fcf.jpg temp/1738924147_3224038_1335028536_2be932d2966c0ee1a0c2862fec845795.jpg temp/1738924147_3224038_1335028532_a1efa0fab172d41649107cf26709bb52.jpg temp/1738924147_3224038_1335028528_eb28631a1d8d98a5f33d79d61bbead59.jpg temp/1738924147_3224038_1335028524_e44867ef883accc61e936910373342b1.jpg temp/1738924147_3224038_1335028448_0ad92bb902b7a899dd43b14e8ab036be.jpg temp/1738924147_3224038_1335028447_ed1f4f8116fae76f9958ff0df4a4da87.jpg temp/1738924147_3224038_1335028446_2f82cb68347b313898d15588dab94912.jpg temp/1738924147_3224038_1335028438_d92d6146fe0c17064cfcd3fd37056a25.jpg temp/1738924147_3224038_1335028428_0e35146e7bd46409ebb0f0b3d3e000dd.jpg temp/1738924147_3224038_1335028390_5a7615de55c1182a31517d406a9d7ccb.jpg temp/1738924147_3224038_1335028257_e528cb346ece07ac1f043b755e09320e.jpg temp/1738924147_3224038_1335028254_35774c1bcb7dac831610ccc983fe3622.jpg temp/1738924147_3224038_1335028236_a50c74e4924a9386aef9cdfb213c9ca9.jpg temp/1738924147_3224038_1335028232_ab09a5cc1960ecbf9e2871191c996e6b.jpg temp/1738924147_3224038_1335028227_23528bdcd4a0cc73b821ef83e43b8a3a.jpg temp/1738924147_3224038_1335028223_0ef4102577b115194eb593b7e7be411e.jpg temp/1738924147_3224038_1335028207_c5dfc7ca02c02bc80faf35ce916177c3.jpg temp/1738924147_3224038_1335028204_c6d4bcd15f47b29e218f16d0f0ba243f.jpg temp/1738924147_3224038_1335028200_6500b4a402d885f463418397ecadebf8.jpg temp/1738924147_3224038_1335028196_d5961a061a2f29dd6d8d459fb70111ef.jpg temp/1738924147_3224038_1335028190_a981ddddbbb58c66eb04a6f4b36c20cf.jpg temp/1738924147_3224038_1335028186_b900c7fd64f36e274e73e8121ed5b89a.jpg temp/1738924147_3224038_1335028137_9cfc4efbe547355ce6535d801604b92a.jpg temp/1738924147_3224038_1335028134_e0e346677e95134e85b6e3c61c891133.jpg temp/1738924147_3224038_1335028131_e243830f8270478ea0a03d3b2c3f1e63.jpg temp/1738924147_3224038_1335028123_d0d7a9116fb63381e01314025d28d9ac.jpg temp/1738924147_3224038_1335028119_28b4e963ea91016923b00385cb0071a5.jpg temp/1738924147_3224038_1335028115_c5a7a6af37ddfcbe7d24b7e1c3535f37.jpg temp/1738924147_3224038_1335028099_1302d5645e96a3daff4efb9c4e69cf90.jpg temp/1738924147_3224038_1335028095_45c75802678ba62ee42c5856aada5488.jpg temp/1738924147_3224038_1335028092_4106d4d4e7b4aaa8109d03ab9d3ea5d4.jpg temp/1738924147_3224038_1335028090_a00212ce0d5a121b0f9302ffe145bbaa.jpg temp/1738924147_3224038_1335028087_7e37537108fd25057174667b3591d9f5.jpg temp/1738924147_3224038_1335028085_9b6a869c38c080c67269e2c6b97bbe95.jpg temp/1738924147_3224038_1335028082_63d9685927731c6cb7978d398618e135.jpg temp/1738924147_3224038_1335028080_f5d770742281e828cac1f12d82eaf43e.jpg temp/1738924147_3224038_1335028078_6189e8f777b5d8d2a2270b6210ef9a67.jpg temp/1738924147_3224038_1335028076_27acb7244eb85ba6bcefe6011484eb96.jpg temp/1738924147_3224038_1335028075_1d3d69dba024aa4a1a3f9f5f5c41e428.jpg temp/1738924147_3224038_1335028072_f02228989581ee56d8b9464494426b56.jpg temp/1738924147_3224038_1335028042_6089aac664ff3ceb0350cd049b1c8429.jpg temp/1738924147_3224038_1335028039_547b299666b4932d00432b0cfdeee999.jpg temp/1738924147_3224038_1335028036_b1b8b817b556381e11d25a7a8c1eed77.jpg temp/1738924147_3224038_1335028033_2614eacae5c2905dce3fb7b3449b6bb0.jpg temp/1738924147_3224038_1335028030_513aaf495500791e14d9defbfc892a51.jpg temp/1738924147_3224038_1335028027_f385841f2d2aa63bf88f24e9b1c1e004.jpg Found 300 images belonging to 1 classes. begin to do the prediction : ERROR in datou_step_exec, will save and exit ! Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above. [[{{node model/module/StatefulPartitionedCall/StatefulPartitionedCall/StatefulPartitionedCall/conv1/StatefulPartitionedCall/Conv2D}}]] [Op:__inference_predict_function_34490] Function call stack: predict_function File "/home/admin/workarea/git/Velours/python/mtr/datou/datou_lib.py", line 2329, in datou_exec output = datou_step_exec(sNext, args, cache, context, map_info, verbose, mtr_user_id) File "/home/admin/workarea/git/Velours/python/mtr/datou/datou_lib.py", line 2523, in datou_step_exec return lib_process.datou_step_tfhub2(param, json_param, args, cache, context, map_info, verbose) File "/home/admin/workarea/git/Velours/python/mtr/datou/lib_step_exec/lib_step_process.py", line 3146, in datou_step_tfhub2 classes, outputs, features = this_model.predict_image_paths(list_paths, keep_aspect_ratio=keep_aspect_ratio, File "/home/admin/workarea/git/Velours/python/mtr/tfhub2/evaluate.py", line 288, in predict_image_paths Y_pred, F_pred = self.model.predict(valid_generator, validation_steps) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/training.py", line 88, in _method_wrapper return method(self, *args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/training.py", line 1268, in predict tmp_batch_outputs = predict_function(iterator) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py", line 580, in __call__ result = self._call(*args, **kwds) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py", line 650, in _call return self._concrete_stateful_fn._filtered_call(canon_args, canon_kwds) # pylint: disable=protected-access File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 1661, in _filtered_call return self._call_flat( File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 1745, in _call_flat return self._build_call_outputs(self._inference_function.call( File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 593, in call outputs = execute.execute( File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/execute.py", line 59, in quick_execute tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name, [1335031573, 1335031571, 1335031569, 1335031568, 1335031567, 1335031566, 1335031562, 1335031561, 1335031558, 1335031555, 1335031550, 1335031476, 1335031400, 1335031397, 1335031394, 1335031393, 1335031389, 1335031386, 1335031366, 1335031360, 1335031356, 1335031344, 1335031343, 1335031341, 1335031328, 1335031324, 1335031319, 1335031315, 1335031312, 1335031310, 1335031293, 1335031292, 1335031290, 1335031287, 1335031286, 1335031284, 1335031267, 1335031264, 1335031262, 1335031259, 1335031256, 1335031254, 1335031235, 1335031233, 1335031230, 1335031225, 1335031221, 1335031220, 1335031049, 1335030946, 1335030826, 1335030821, 1335030814, 1335030787, 1335030762, 1335030758, 1335030752, 1335030724, 1335030719, 1335030716, 1335030674, 1335030617, 1335030614, 1335030611, 1335030609, 1335030608, 1335030590, 1335030587, 1335030578, 1335030574, 1335030569, 1335030565, 1335030483, 1335030479, 1335030474, 1335030465, 1335030457, 1335030455, 1335030434, 1335030431, 1335030373, 1335030342, 1335030261, 1335030251, 1335030226, 1335030223, 1335030222, 1335030221, 1335030219, 1335030216, 1335030139, 1335030135, 1335030132, 1335030131, 1335030128, 1335030126, 1335030103, 1335030102, 1335030101, 1335030094, 1335030091, 1335030088, 1335029918, 1335029916, 1335029915, 1335029912, 1335029908, 1335029905, 1335029889, 1335029887, 1335029886, 1335029884, 1335029881, 1335029878, 1335029834, 1335029801, 1335029742, 1335029721, 1335029703, 1335029679, 1335029642, 1335029608, 1335029568, 1335029534, 1335029462, 1335029416, 1335029312, 1335029311, 1335029310, 1335029309, 1335029306, 1335029303, 1335029297, 1335029296, 1335029294, 1335029292, 1335029289, 1335029288, 1335029285, 1335029284, 1335029281, 1335029280, 1335029277, 1335029275, 1335029196, 1335029193, 1335029192, 1335029188, 1335029187, 1335029183, 1335029182, 1335029177, 1335029176, 1335029172, 1335029170, 1335029165, 1335029145, 1335029142, 1335029139, 1335029137, 1335029134, 1335029132, 1335029121, 1335029119, 1335029116, 1335029113, 1335029110, 1335029107, 1335029092, 1335029045, 1335029040, 1335029037, 1335029030, 1335029000, 1335028953, 1335028951, 1335028948, 1335028945, 1335028942, 1335028939, 1335028913, 1335028907, 1335028906, 1335028902, 1335028900, 1335028890, 1335028888, 1335028884, 1335028873, 1335028871, 1335028869, 1335028868, 1335028839, 1335028837, 1335028834, 1335028832, 1335028829, 1335028828, 1335028826, 1335028825, 1335028823, 1335028821, 1335028817, 1335028815, 1335028792, 1335028788, 1335028784, 1335028780, 1335028776, 1335028773, 1335028762, 1335028761, 1335028760, 1335028759, 1335028756, 1335028753, 1335028740, 1335028736, 1335028732, 1335028729, 1335028726, 1335028721, 1335028705, 1335028703, 1335028701, 1335028700, 1335028681, 1335028680, 1335028677, 1335028676, 1335028673, 1335028672, 1335028670, 1335028669, 1335028656, 1335028655, 1335028652, 1335028650, 1335028647, 1335028646, 1335028644, 1335028643, 1335028616, 1335028615, 1335028613, 1335028611, 1335028597, 1335028594, 1335028590, 1335028586, 1335028576, 1335028570, 1335028542, 1335028539, 1335028536, 1335028532, 1335028528, 1335028524, 1335028448, 1335028447, 1335028446, 1335028438, 1335028428, 1335028390, 1335028257, 1335028254, 1335028236, 1335028232, 1335028227, 1335028223, 1335028207, 1335028204, 1335028200, 1335028196, 1335028190, 1335028186, 1335028137, 1335028134, 1335028131, 1335028123, 1335028119, 1335028115, 1335028099, 1335028095, 1335028092, 1335028090, 1335028087, 1335028085, 1335028082, 1335028080, 1335028078, 1335028076, 1335028075, 1335028072, 1335028042, 1335028039, 1335028036, 1335028033, 1335028030, 1335028027] begin to insert list_values into mtr_datou_result : length of list_values in save_final : 300 time used for this insertion : 0.25712013244628906 save_final ERROR in last step tfhub_classification2, Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above. [[{{node model/module/StatefulPartitionedCall/StatefulPartitionedCall/StatefulPartitionedCall/conv1/StatefulPartitionedCall/Conv2D}}]] [Op:__inference_predict_function_34490] Function call stack: predict_function time spend for datou_step_exec : 15.719128370285034 time spend to save output : 0.26669883728027344 total time spend for step 0 : 15.985827207565308 need to delete datou_research and reload, so keep current state 1 caffe_path_current : About to save ! 2 After save, about to update current ! 15.64user 5.48system 1:27.63elapsed 24%CPU (0avgtext+0avgdata 2159428maxresident)k 587216inputs+1254104outputs (13major+659297minor)pagefaults 0swaps