python /home/admin/mtr/script_for_cron.py -j default -m 20 -a 'python3 ~/workarea/git/Velours/python/prod/datou.py -j batch_current -C 2562432' -s traitement_4892 -M 0 -S 0 -U 100,80,95 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/home/admin/workarea/git/apy', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3603119 load datou : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : step 0 init_dummy_multi_datou is not linked in the step_by_step architecture ! WARNING : step 1294 init_dummy_multi_datou is not linked in the step_by_step architecture ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : (photo_id, hashtag_id, score_max) was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec no input labels no input values updating current state to 1 we have a portfolio with more photos than limit : 7456>1000 please execute split_portfolio.py -i 20244820 -l 1000 size over we load limit photo not treated list_input_json: {} Current got : datou_id : 4892, datou_cur_ids : ['2562432'] with mtr_portfolio_ids : ['20244820'] and first list_photo_ids : [] new path : /proc/3603119/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! DataTypes for each output/input checked ! List Step Type Loaded in datou : tfhub_classification2, argmax over limit max, limiting to limit_max 300 list_input_json : {} origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 300 ; length of list_pids : 300 ; length of list_args : 300 time to download the photos : 64.27478122711182 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 2 step1:tfhub_classification2 Fri Feb 7 12:33:14 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou_step TFHub with tf2 ! nombre de thcls : 2 we are using the classfication for multi_thcl [3513, 3890] begin to check gpu status inside check gpu memory 2025-02-07 12:33:20.598531: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-07 12:33:21.662552: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-07 12:33:21.663221: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-07 12:33:21.668082: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-07 12:33:21.718396: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-07 12:33:21.719896: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-07 12:33:21.804671: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-07 12:33:21.822333: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-07 12:33:22.038982: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-07 12:33:22.040795: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-07 12:33:22.046933: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-07 12:33:22.095129: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-07 12:33:22.097088: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fd934000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-07 12:33:22.097118: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-07 12:33:22.246230: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x249de050 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-07 12:33:22.246278: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-07 12:33:22.250103: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-07 12:33:22.250831: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-07 12:33:22.250873: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-07 12:33:22.250918: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-07 12:33:22.250956: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-07 12:33:22.250986: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-07 12:33:22.251019: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-07 12:33:22.251051: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-07 12:33:22.252618: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-07 12:33:22.253081: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-07 12:33:22.254421: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-07 12:33:22.254449: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-07 12:33:22.254463: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-07 12:33:22.256293: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3096 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) l 3637 free memory gpu now : 3137 max_wait_temp : 1 max_wait : 5 1 Physical GPUs, 1 Logical GPUs tagging for thcl : 3513 To do loadFromThcl(), then load ParamDescType : thcl3513 thcls : [{'id': 3513, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2_tf', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 4557, 'photo_desc_type': 5767, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] thcl {'id': 3513, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2_tf', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 4557, 'photo_desc_type': 5767, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'} Update svm_hashtag_type_desc : 5767 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5767, 'Rungis_amount_dechets_fall_2018_v2_tf', 2048, 2048, 'Rungis_amount_dechets_fall_2018_v2_tf', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 3, 16, 15, 52, 10), datetime.datetime(2023, 3, 16, 15, 52, 10)) model_name : Rungis_amount_dechets_fall_2018_v2_tf model_param file didn't exist model_name : Rungis_amount_dechets_fall_2018_v2_tf model_type : tf_classification2 list file need : ['Confusion_Matrix.png', 'Precision_Recall_05102018_Papier_non_papier_dense.jpg', 'Precision_Recall_05102018_Papier_non_papier_peu_dense.jpg', 'Precision_Recall_05102018_Papier_non_papier_presque_vide.jpg', 'Precision_Recall_05102018_Papier_non_papier_tres_dense.jpg', 'Precision_Recall_05102018_Papier_non_papier_tres_peu_dense.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00001', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file exist in s3 : ['Confusion_Matrix.png', 'Precision_Recall_05102018_Papier_non_papier_dense.jpg', 'Precision_Recall_05102018_Papier_non_papier_peu_dense.jpg', 'Precision_Recall_05102018_Papier_non_papier_presque_vide.jpg', 'Precision_Recall_05102018_Papier_non_papier_tres_dense.jpg', 'Precision_Recall_05102018_Papier_non_papier_tres_peu_dense.jpg', 'Result_Summary.txt', 'checkpoint', 'model_checkpoint.ckpt.data-00000-of-00001', 'model_checkpoint.ckpt.data-00000-of-00002', 'model_checkpoint.ckpt.data-00001-of-00002', 'model_checkpoint.ckpt.index', 'model_weights.h5'] file manque in s3 : [] 2025-02-07 12:33:29.890161: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.02G (3246391296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 12:33:29.891383: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.72G (2921752064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/Confusion_Matrix.png size_local : 67810 size in s3 : 67810 create time local : 2023-10-30 16:21:37 create time in s3 : 2023-10-30 14:09:29 Confusion_Matrix.png already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/Precision_Recall_05102018_Papier_non_papier_dense.jpg size_local : 73949 size in s3 : 73949 create time local : 2023-10-30 16:21:38 create time in s3 : 2023-10-30 14:09:30 Precision_Recall_05102018_Papier_non_papier_dense.jpg already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/Precision_Recall_05102018_Papier_non_papier_peu_dense.jpg size_local : 85572 size in s3 : 85572 create time local : 2023-10-30 16:21:38 create time in s3 : 2023-10-30 14:09:39 Precision_Recall_05102018_Papier_non_papier_peu_dense.jpg already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/Precision_Recall_05102018_Papier_non_papier_presque_vide.jpg size_local : 72361 size in s3 : 72361 create time local : 2023-10-30 16:21:38 create time in s3 : 2023-10-30 14:09:37 Precision_Recall_05102018_Papier_non_papier_presque_vide.jpg already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/Precision_Recall_05102018_Papier_non_papier_tres_dense.jpg size_local : 83567 size in s3 : 83567 create time local : 2023-10-30 16:21:38 create time in s3 : 2023-10-30 14:09:48 Precision_Recall_05102018_Papier_non_papier_tres_dense.jpg already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/Precision_Recall_05102018_Papier_non_papier_tres_peu_dense.jpg size_local : 71611 size in s3 : 71611 create time local : 2023-10-30 16:21:38 create time in s3 : 2023-10-30 14:09:29 Precision_Recall_05102018_Papier_non_papier_tres_peu_dense.jpg already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/Result_Summary.txt size_local : 1058 size in s3 : 1058 create time local : 2023-10-30 16:21:38 create time in s3 : 2023-10-30 14:09:30 Result_Summary.txt already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/checkpoint size_local : 99 size in s3 : 99 create time local : 2023-10-30 16:21:38 create time in s3 : 2023-10-30 14:09:36 checkpoint already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/model_checkpoint.ckpt.data-00000-of-00001 size_local : 188538519 size in s3 : 188538519 create time local : 2023-10-30 16:21:41 create time in s3 : 2023-10-30 14:09:40 model_checkpoint.ckpt.data-00000-of-00001 already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/model_checkpoint.ckpt.data-00000-of-00002 size_local : 216572 size in s3 : 216572 create time local : 2023-10-30 16:21:41 create time in s3 : 2023-03-16 14:52:09 model_checkpoint.ckpt.data-00000-of-00002 already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/model_checkpoint.ckpt.data-00001-of-00002 size_local : 32279708 size in s3 : 32279708 create time local : 2023-10-30 16:21:42 create time in s3 : 2023-03-16 14:52:07 model_checkpoint.ckpt.data-00001-of-00002 already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/model_checkpoint.ckpt.index size_local : 28001 size in s3 : 28001 create time local : 2023-10-30 16:21:42 create time in s3 : 2023-10-30 14:09:30 model_checkpoint.ckpt.index already exist and didn't need to update /data/models_weight/Rungis_amount_dechets_fall_2018_v2_tf/model_weights.h5 size_local : 94501976 size in s3 : 94501976 create time local : 2023-10-30 16:21:44 create time in s3 : 2023-10-30 14:09:37 model_weights.h5 already exist and didn't need to update desc size : 2048 Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= module (KerasLayer) (None, 2048) 23561152 _________________________________________________________________ Rungis_amount_dechets_fall_2 (None, 5) 10245 ================================================================= Total params: 23,571,397 Trainable params: 10,245 Non-trainable params: 23,561,152 _________________________________________________________________ Loading Weights... time used to create the model : 11.858478307723999 time used to load_weights : 0.5628044605255127 0it [00:00, ?it/s] 9it [00:00, 87.01it/s] 37it [00:00, 192.57it/s] 73it [00:00, 265.99it/s] 100it [00:00, 223.27it/s]temp/1738927929_3603119_1334982433_e09e0d0b0c6d77d3026fda5c9486e30c.jpg temp/1738927929_3603119_1334982429_a1d754bfd73e019b38a6d6054cf81ed5.jpg temp/1738927929_3603119_1334982426_d76e073370bf1d47c795682bd0d1a41f.jpg temp/1738927929_3603119_1334982424_d364be10086ac513436c298e2f5bd36e.jpg temp/1738927929_3603119_1334982423_dbd2cf90a9fd8ea14aa41ed3edeed6a6.jpg temp/1738927929_3603119_1334982421_e96bb6b5e9a4428a24d424c0ac8fd472.jpg temp/1738927929_3603119_1334982392_a464e4123c12ec7a8945109037e022f1.jpg temp/1738927929_3603119_1334982391_7d16412fae202d056e29a8f793ae18bf.jpg temp/1738927929_3603119_1334982390_7b1ff34b786747fda90935d405dd543a.jpg temp/1738927929_3603119_1334982387_a3219495672671894c659c932f8d02d1.jpg temp/1738927929_3603119_1334982384_aa12562350271207417b3abaeb693c24.jpg temp/1738927929_3603119_1334982376_1fe294c058693175d9f0282f111981b0.jpg temp/1738927929_3603119_1334982100_57e145f3c910c7c3c96f6feedae37de9.jpg temp/1738927929_3603119_1334982099_a1ddf6b19ca8e53c78365100af45a951.jpg temp/1738927929_3603119_1334982095_6a11682b071996645add52d3edad5994.jpg temp/1738927929_3603119_1334982087_c7b4a1917805ef93dc91f725e3f03c8f.jpg temp/1738927929_3603119_1334982065_f02a87026abebe761b61a65aa2339843.jpg temp/1738927929_3603119_1334982062_4daaeb571f2f10ce9812700eec7b2cd0.jpg temp/1738927929_3603119_1334982022_7cb3e9534aed34ebcedd2a7b62640dc1.jpg temp/1738927929_3603119_1334981955_91a7170846e3b435d2d6d05953489280.jpg temp/1738927929_3603119_1334981918_43320dfdc34730cf1b590e86dc3bffb8.jpg temp/1738927929_3603119_1334981915_2ba60f3cf98dd3cb8e439edfc248f231.jpg temp/1738927929_3603119_1334981908_23c6feec874a32b39cdae56094732ea4.jpg temp/1738927929_3603119_1334981906_20541c61d73ce851d7bd30730048d885.jpg temp/1738927929_3603119_1334981276_dd4f8d637fdcfc22d5008a5b362a56ae.jpg temp/1738927929_3603119_1334981205_d7fbd6fa3f3add45f22f2d6422efbb13.jpg temp/1738927929_3603119_1334981142_9d0b7d5f40f885ec2aefa68d9df3d3c0.jpg temp/1738927929_3603119_1334981077_8b2ab4c35f777344cf7c00c4121acaed.jpg temp/1738927929_3603119_1334981062_59cfb2b61bcf6c84116e0c576267a960.jpg temp/1738927929_3603119_1334981057_aa83992ebbf3dca579605570172433e6.jpg temp/1738927929_3603119_1334981030_14f5e5f8283f057c2863e96779d7920b.jpg temp/1738927929_3603119_1334981027_4e631a3fff424d43f90ab380993697e9.jpg temp/1738927929_3603119_1334981024_cdc91c2e687bbe0fd13e47bb74dcd280.jpg temp/1738927929_3603119_1334981021_1845ee2616e4faa358de80c47d32bad2.jpg temp/1738927929_3603119_1334981018_bdeee8e86b63961993c7d7c031832a24.jpg temp/1738927929_3603119_1334981014_ae9bbfeafbbe94e92bb178f8aa56cf73.jpg temp/1738927929_3603119_1334980964_7a5389c07300b04f93a7d77bc48f614c.jpg temp/1738927929_3603119_1334980961_2cbd2efb5ace8e5162542cf08c8a50d7.jpg temp/1738927929_3603119_1334980957_47be0eea0debdb7af51499987eb7aefd.jpg temp/1738927929_3603119_1334980953_d18e6ef383bffec9f2b9136b54356d05.jpg temp/1738927929_3603119_1334980939_09f0bd01935e51f6055ec00cc6277a97.jpg temp/1738927929_3603119_1334980930_1e88ed6b43df05d3393b7780bb3a9b70.jpg temp/1738927929_3603119_1334980878_11e242b22135660b6a16e6a966443670.jpg temp/1738927929_3603119_1334980835_9abb280bda51530f62c8673c5e5569ae.jpg temp/1738927929_3603119_1334980826_a749e34237dd27301dbd791d55ada0a8.jpg temp/1738927929_3603119_1334980825_c38bf2e6427e289fe009d4a718ca721b.jpg temp/1738927929_3603119_1334980824_061b97b323d30236451d5f59c60e4973.jpg temp/1738927929_3603119_1334980822_4caa17899fd27eb681afb3982278e7ba.jpg temp/1738927929_3603119_1334980766_2184bfb3ad6881cdbdef79b8d61c369a.jpg temp/1738927929_3603119_1334980762_51c6c029ba55d054e293f904b6c0766c.jpg temp/1738927929_3603119_1334980759_a6ac727c94c3001141e43ede7287ce49.jpg temp/1738927929_3603119_1334980756_e1f6e111fce1567c153b18c80b25818e.jpg temp/1738927929_3603119_1334980753_8e0a919feb57e3bfb87db7e12a3b62f4.jpg temp/1738927929_3603119_1334980750_3edac6b76b0950af958ad4a6a3e75f70.jpg temp/1738927929_3603119_1334980522_29b4def78d5f91830ecf6bb168bbdc84.jpg temp/1738927929_3603119_1334980454_2f66ffc4fa2bb8cab50a30e1bec0987b.jpg temp/1738927929_3603119_1334980366_fc2cc1a0343cfd6e5217a12b801cdee6.jpg temp/1738927929_3603119_1334980249_18ff945623752980ef3028b1e1867d2c.jpg temp/1738927929_3603119_1334980227_69cefced863c14a83cecbf646f117e07.jpg temp/1738927929_3603119_1334980221_19bb7bff6d6fde7f45edd5f974394f7f.jpg temp/1738927929_3603119_1334980170_ba748a9da9c0f3224ab8e920e5c2d8df.jpg temp/1738927929_3603119_1334980167_1613b3bdec6e1a136a41ff58a5ef09db.jpg temp/1738927929_3603119_1334980165_61363cfcbfe39ea4c74cce134907a83f.jpg temp/1738927929_3603119_1334980161_b2da6a53c8a7c9da40d94da915eaa612.jpg temp/1738927929_3603119_1334980157_339e91705a759b4ce3f00f21856fa95b.jpg temp/1738927929_3603119_1334980152_0a2831e499c405c9c00aceb74ff46b39.jpg temp/1738927929_3603119_1334980097_a88ff8020af4aaeae3019b66629c3102.jpg temp/1738927929_3603119_1334980095_9c00e649978db4f51284aca4617cb58c.jpg temp/1738927929_3603119_1334980083_323411ef39a7284564aef4fdd012634f.jpg temp/1738927929_3603119_1334980081_11edc6d395c1d9ca72bc32e7fd91a462.jpg temp/1738927929_3603119_1334980077_87d6ddab672b6eb86c60a64398745cd0.jpg temp/1738927929_3603119_1334980075_257badfdaaab893d87379dc68738f4e7.jpg temp/1738927929_3603119_1334980066_2800dce45ae911a70f07debde3928518.jpg temp/1738927929_3603119_1334980042_536f768e6b808ae02582e80caf0ca52c.jpg temp/1738927929_3603119_1334980030_d8ce407c843a39319796374eadae043f.jpg temp/1738927929_3603119_1334980028_603be9d3caf5ec941948f7330f295605.jpg temp/1738927929_3603119_1334980024_cd3379534e2a0408996957c31d16352c.jpg temp/1738927929_3603119_1334980023_fee6d03dd0fa6d8822b35af62b045f71.jpg temp/1738927929_3603119_1334979907_1c43b439e40ed4481a88f7b98280ca02.jpg temp/1738927929_3603119_1334979902_8a3bb08c772e6abc6e81ecb0c5a90523.jpg temp/1738927929_3603119_1334979897_0be3b77e3c1e81f5004d610512698893.jpg temp/1738927929_3603119_1334979892_93fa7e210f2122a8f2cfcf607fbd6f57.jpg temp/1738927929_3603119_1334979888_99f1bb6db15ffc38a41a85b01370ed9c.jpg temp/1738927929_3603119_1334979885_90c262651f67eb0ba98e4f1e0d38a0f9.jpg temp/1738927929_3603119_1334979849_5673eecd77d0a3e9c1a180ca382c21a0.jpg temp/1738927929_3603119_1334979845_d960c55cfd4fefd54474ceb4b0cd31c1.jpg temp/1738927929_3603119_1334979840_6de0c923e5cd950104b1061a417ceba8.jpg temp/1738927929_3603119_1334979835_769ca12cad0125b1c15c7cdc32e7264c.jpg temp/1738927929_3603119_1334979830_642d218bc38ea9fec8eef76fdcd42ec2.jpg temp/1738927929_3603119_1334979826_7b1ab8609977e28f4c9ffe6418557192.jpg temp/1738927929_3603119_1334979576_27cb99a2f6630300aac9191a445adb90.jpg temp/1738927929_3603119_1334979572_d265c9ab30bf568b662c9e99d12f93bb.jpg temp/1738927929_3603119_1334979569_707caf783a3eb76f93b2a80d743dc28c.jpg temp/1738927929_3603119_1334979565_874a9727a3668e99e06ad06f6b398e66.jpg temp/1738927929_3603119_1334979562_3f92b3199ee607bf81b8f8c0ffea8ba3.jpg temp/1738927929_3603119_1334979557_2b6bfe3f7421a3d4cf18d67fe73162ee.jpg temp/1738927929_3603119_1334979356_1628141f7fd5d16886eebba5c5abbe11.jpg temp/1738927929_3603119_1334979352_b003f302066bfd374023e25c5e797f7e.jpg temp/1738927929_3603119_1334979348_cf31395b553c2cf741f22878bf39c081.jpg temp/1738927929_3603119_1334979344_bfa180256a500661262bfb6b6c440140.jpg temp/1738927929_3603119_1334979342_6a19b2bb6103c2c5d993d290dcae8efe.jpg temp/1738927929_3603119_1334979340_c2b61d635536a4e5be675e74d3a9b7ed.jpg temp/1738927929_3603119_1334979307_e8ef1ce4689e65de94b1b52e9f0320f4.jpg temp/1738927929_3603119_1334979300_719257cdf215aac32ca94f08b1e1fb1d.jpg temp/1738927929_3603119_1334979296_c968939e7617b2f530f1a379e476c93d.jpg temp/1738927929_3603119_1334979291_639cb0ffa9b902e798981d490aac985a.jpg temp/1738927929_3603119_1334979287_1e65117289ba6fc92ac6d095dd47ba12.jpg temp/1738927929_3603119_1334979285_9c208094917cb058ec60915024df232f.jpg temp/1738927929_3603119_1334979194_2d28966bb2ebda470c3104af09b60467.jpg temp/1738927929_3603119_1334979187_d9497e2f3bb925df107f878f2ed70a4d.jpg temp/1738927929_3603119_1334979170_f291153901a21cf029579e98797d988e.jpg temp/1738927929_3603119_1334979168_f31ee87ae8a91e1bc5d9ae3f9a097cd7.jpg temp/1738927929_3603119_1334979151_91fd7ecc3a37c972a9568372e7b0b9b9.jpg 124it [00:00, 130.15it/s] 145it [00:00, 139.98it/s] 163it [00:01, 141.20it/s] 180it [00:01, 139.22it/s] 196it [00:01, 141.58it/s] 212it [00:01, 140.35it/s]temp/1738927929_3603119_1334979149_7dc9504b5af222e0adfb5004f7789cc1.jpg temp/1738927929_3603119_1334979100_079a105f24990b90c041ed2011241111.jpg temp/1738927929_3603119_1334979097_e6814ae72f5e05ac058f615d38b6fe87.jpg temp/1738927929_3603119_1334979096_bca12c7dc643a43ca046c8443025aa95.jpg temp/1738927929_3603119_1334979094_1025950c3a99631ff8682566d4970a6e.jpg temp/1738927929_3603119_1334979061_823de0b959be5a0ac1de0c4b33770cdf.jpg temp/1738927929_3603119_1334979058_db8d5b341f3b34b4ce9ea7ce4aa889ad.jpg temp/1738927929_3603119_1334978992_1af5a2eab1afa9c317f8c55bcd12c17d.jpg temp/1738927929_3603119_1334978989_23ea28fc6dcec1a20f7eb3dbdde3acd3.jpg temp/1738927929_3603119_1334978974_c13370c4e298956aabaadf131fadbacb.jpg temp/1738927929_3603119_1334978934_8815b145216182c1e1149d11f0f0e9f7.jpg temp/1738927929_3603119_1334978929_40c7a7dd7526ff095f46a3980214461e.jpg temp/1738927929_3603119_1334978784_c26880fa5273cdd86e0d51baa4fca277.jpg temp/1738927929_3603119_1334978624_c8b1624b4d1b49115b6a743683298007.jpg temp/1738927929_3603119_1334978622_7a246c58ac872021f7600cc72a86fbd5.jpg temp/1738927929_3603119_1334978621_1016fe81cf6f74aaafe0c3184e819a99.jpg temp/1738927929_3603119_1334978617_08a1866212e61335c7aa01cf49967e20.jpg temp/1738927929_3603119_1334978615_fe41455d5c16fdc13db3f0b2049e9773.jpg temp/1738927929_3603119_1334978607_a09c342b23ae6e677a2812bb21593604.jpg temp/1738927929_3603119_1334978365_240254a68adef1d9388992b191a50557.jpg temp/1738927929_3603119_1334978361_f37ea3cacd6b244187075cb2a1923afb.jpg temp/1738927929_3603119_1334978219_41b1c5e59f8c2482830c12e35c55b300.jpg temp/1738927929_3603119_1334978176_90d3f1f9416f275fc6a13bfd4fa504f5.jpg temp/1738927929_3603119_1334978138_b35115adb78ab90e4e72c14ed03a35c6.jpg temp/1738927929_3603119_1334978105_814dbb60d5c567c57f3aa674e6064624.jpg temp/1738927929_3603119_1334977633_d1fdd617fa89b8b6f8d5fbaa5067beff.jpg temp/1738927929_3603119_1334977448_cbc4c297f3365412579f616212e2ac27.jpg temp/1738927929_3603119_1334977363_1c735585c02cda7d9c7f9ff0750b710c.jpg temp/1738927929_3603119_1334977317_b99b7f64769df10b69ce1cf349c8ce0a.jpg temp/1738927929_3603119_1334977315_087c0e45dc997519280d5dd118bd2c47.jpg temp/1738927929_3603119_1334977312_cf7c8c457a817937f60a76bb1d19d4c6.jpg temp/1738927929_3603119_1334976798_c2fc6a162ecda412f9678fa05da46fac.jpg temp/1738927929_3603119_1334976797_82a17ee08dcf3772537ffb4121324f66.jpg temp/1738927929_3603119_1334976795_6e48144b6988d8ba3c17f2e1de663fac.jpg temp/1738927929_3603119_1334976792_375b4a0620d837678c0e0cea24696dcc.jpg temp/1738927929_3603119_1334976770_c888e020e88e1f63c038f5f0f52afe81.jpg temp/1738927929_3603119_1334976765_eebc4867a4bc748e0823ee5f3ed97de0.jpg temp/1738927929_3603119_1334976715_2e385787008e85cc22cf171bfb133d91.jpg temp/1738927929_3603119_1334976712_e9a2fa9d10f3c89aaffa1988b0c02eb4.jpg temp/1738927929_3603119_1334976708_768dff4aecacab24925f7121e7b585fd.jpg temp/1738927929_3603119_1334976707_a0c37ba17640f32cb1d28fd2682ae69b.jpg temp/1738927929_3603119_1334976697_8d9f078ccdc5b9e17ee2ccdca60af37a.jpg temp/1738927929_3603119_1334976696_de4f089d53a0a284aa0a6dc85025d501.jpg temp/1738927929_3603119_1334976684_b047645ca037e5e2a9ca5c7682ef390c.jpg temp/1738927929_3603119_1334976683_888bc9fce0dd56bd247a6822c7a91d69.jpg temp/1738927929_3603119_1334976680_f5dc752a4d96764534c2ae73b9df5dbb.jpg temp/1738927929_3603119_1334976678_0da41f7226d1ebfa655f74591810d2f6.jpg temp/1738927929_3603119_1334976677_0c69d4a758d99a16dbd19e10ad1c9ed1.jpg temp/1738927929_3603119_1334976675_f072883bfd72370d6c2d0e016204742c.jpg temp/1738927929_3603119_1334976650_62a7e3988ff2c2650b27539b9abc8960.jpg temp/1738927929_3603119_1334976647_ab95ee30e32960df6abad0a046b6c763.jpg temp/1738927929_3603119_1334976646_9e978ef0ed2313ede1ca431c0e740797.jpg temp/1738927929_3603119_1334976630_60773b72568a23bbd1d1a3e19e93271a.jpg temp/1738927929_3603119_1334976558_e7ddd16060fcecbb77ed60f8a22a33b9.jpg temp/1738927929_3603119_1334976503_88e163d423829357cabdf7162fd4857a.jpg temp/1738927929_3603119_1334976478_2839dc682a3d225bb693511c2d3f1ddc.jpg temp/1738927929_3603119_1334976473_becdcf7321eb1addb5dce726cd5884d9.jpg temp/1738927929_3603119_1334976471_81a4c59e50dbf6e669133e76d1bae80d.jpg temp/1738927929_3603119_1334976468_c0e33927d30f5daff66f7ede3477df27.jpg temp/1738927929_3603119_1334976466_5c6100a299aa1d09c9285c983d86e407.jpg temp/1738927929_3603119_1334976460_df97f8065b9b33c3b403191cb46b7ae9.jpg temp/1738927929_3603119_1334976393_22744c758376aa641047f363978af69e.jpg temp/1738927929_3603119_1334976390_b606bc8a366d04814bb6a1121dce27b0.jpg temp/1738927929_3603119_1334976370_4f8cc69dba426b8c55a927eab2a80937.jpg temp/1738927929_3603119_1334976328_cef7763d522f8c2a11308f0321b12ca7.jpg temp/1738927929_3603119_1334976324_856eaf624c6ec5b563c89f97edcff665.jpg temp/1738927929_3603119_1334976273_5d2c336d83c1b7eabfb28ce0088cf84d.jpg temp/1738927929_3603119_1334976153_0046172beb7d21ddea1bef601d5e94ea.jpg temp/1738927929_3603119_1334976147_cae0bf42b988394506ca6849a9df2a1d.jpg temp/1738927929_3603119_1334976145_6f154cb25e5409b5cd2c582dc4747de2.jpg temp/1738927929_3603119_1334976144_5174c1748511e198ef816864efc52f7c.jpg temp/1738927929_3603119_1334976143_6c0c7f28fdcc83957c4e856ed706621a.jpg temp/1738927929_3603119_1334976141_4f38a904daa36d1217923080500c6613.jpg temp/1738927929_3603119_1334976120_117710247927f6db42ffd522bba9d8e7.jpg temp/1738927929_3603119_1334976118_30844fcb20f548a35194055c2a08bb0f.jpg temp/1738927929_3603119_1334976117_3e2f52ac80842f14f51a59fa151ab02c.jpg temp/1738927929_3603119_1334976116_e69e86ebcfe15b2ef3648b5a613b53a2.jpg temp/1738927929_3603119_1334976114_cdb31b4f4e0f5a8430e24d2bffd2c584.jpg temp/1738927929_3603119_1334976111_cdcbe042d78de18ef5eed8f252c55052.jpg temp/1738927929_3603119_1334975999_5fd8a72911409dcbba1c606d65f5974e.jpg temp/1738927929_3603119_1334975976_2f72e4abfbe16e60506fd7aa03810c9e.jpg temp/1738927929_3603119_1334975974_9bae86243d5852641d2ddcdc157aeed5.jpg temp/1738927929_3603119_1334975961_e7481c45001267f97ddb464fcbcc4d51.jpg temp/1738927929_3603119_1334975959_a3df75a4f89d321b05ac648c2bd498ec.jpg temp/1738927929_3603119_1334975941_9f28a3a664091553882693a7f4753b58.jpg temp/1738927929_3603119_1334975517_45e0b166a3f8d85ca1438b4288cc72c7.jpg temp/1738927929_3603119_1334975378_4b8a7ea03522f39e702a2ead652875a1.jpg temp/1738927929_3603119_1334975375_bde1ff3fa30820c202b472bee8fdcb9d.jpg temp/1738927929_3603119_1334975374_4c73ad532a46a4edf71c1b41b63e853a.jpg temp/1738927929_3603119_1334975371_1208281b007014305ed4479ae024679b.jpg temp/1738927929_3603119_1334975348_82c1650602a6879e6acc0606418c88fa.jpg temp/1738927929_3603119_1334975320_3ad140b33d88250c143f3197dcedfcaf.jpg temp/1738927929_3603119_1334975318_471b5c24dd346ce7d1d9300e98b474f8.jpg temp/1738927929_3603119_1334975316_4615dd925bbd25df0ec7e00c6196275b.jpg temp/1738927929_3603119_1334975314_60bd9f7fc1b27fbdba3e13574c6a504b.jpg temp/1738927929_3603119_1334975312_d8e43517993cff1e668b9ed352451479.jpg temp/1738927929_3603119_1334975311_c82fa8969e0bcc0f9001038b6ffa188e.jpg temp/1738927929_3603119_1334975267_464bea9f088de25517be5ce6f8d0d5be.jpg temp/1738927929_3603119_1334975262_1b6a34fd156064f5410473c9a4e57025.jpg temp/1738927929_3603119_1334975260_2a55dc4f84405e727488795ba91447a0.jpg temp/1738927929_3603119_1334975258_963ac9e1b41fd16d29941dbf002a39d2.jpg temp/1738927929_3603119_1334975256_467b0c3e4adfda93698560fb2261ab35.jpg temp/1738927929_3603119_1334975254_e99898a5173ca5f84714152c4690aa50.jpg temp/1738927929_3603119_1334975242_829709abbe820473c740c2e9726df9ee.jpg temp/1738927929_3603119_1334975240_b5b82dce201b6e27d8e950dc6baa7f44.jpg temp/1738927929_3603119_1334975238_eaf856d338c6ff6269818d099fff0359.jpg temp/1738927929_3603119_1334975235_0161dc0faaa0537b879e84b68b582433.jpg temp/1738927929_3603119_1334975232_4eda04e8850b2992564db34015068e6a.jpg temp/1738927929_3603119_1334975229_23d7f8f636856435723e581375a0f444.jpg temp/1738927929_3603119_1334975209_fe7f978531af36f431cc4771690f0fa1.jpg temp/1738927929_3603119_1334975206_f90481709ee1b02ce54d7cc706833f29.jpg temp/1738927929_3603119_1334975056_1788e1e2760a4075880d28d0099694d8.jpg temp/1738927929_3603119_1334975045_30a5310fa9da270aef500bc2ffecd522.jpg 232it [00:01, 155.07it/s] 267it [00:01, 206.80it/s] 289it [00:01, 165.00it/s] 300it [00:01, 163.84it/s]2025-02-07 12:33:39.164412: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-07 12:33:39.785316: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-07 12:33:40.196851: E tensorflow/stream_executor/cuda/cuda_dnn.cc:329] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR 2025-02-07 12:33:40.210157: E tensorflow/stream_executor/cuda/cuda_dnn.cc:329] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR temp/1738927929_3603119_1334975043_0d0bb097c008e696f766c22d1278cb44.jpg temp/1738927929_3603119_1334975041_e91f3eabe13e6f472b9f3cc1aac9508d.jpg temp/1738927929_3603119_1334974977_4dadeb59619062b5cd7a2ca5291464b5.jpg temp/1738927929_3603119_1334974973_7abae44e75cca0e1592ca080635f44cd.jpg temp/1738927929_3603119_1334974972_685dc8eeefab28d8333ad53081cbef21.jpg temp/1738927929_3603119_1334974963_a3215e3cb792089574ddf6c969eda50f.jpg temp/1738927929_3603119_1334974906_15c0829d036b8e3f16ca7f45f215cb78.jpg temp/1738927929_3603119_1334974902_81671e1daa4e3c718250cb09e95637eb.jpg temp/1738927929_3603119_1334974716_5433f269d4257019fd5a36ec7fb9e0d3.jpg temp/1738927929_3603119_1334974714_e7314b7a26b841dcb95dc5d0ebb29409.jpg temp/1738927929_3603119_1334974712_032a0f996b7d22af08a9fcd24e1219bb.jpg temp/1738927929_3603119_1334974710_4080feadfd0305b08030153a81c84213.jpg temp/1738927929_3603119_1334974707_e16f44783f9b09ecaf59ce7c9f33352c.jpg temp/1738927929_3603119_1334974705_57c7e6b1680459ab8db7abf616ae466f.jpg temp/1738927929_3603119_1334974657_88e3816fa554aaace062079ea19aafda.jpg temp/1738927929_3603119_1334974645_be108d29bd4e873c9320dbed24f9a66e.jpg temp/1738927929_3603119_1334974549_43af289c52530abcf5f4866be246ca2d.jpg temp/1738927929_3603119_1334974469_b01659b79b3f32369716f2bb9eb41413.jpg temp/1738927929_3603119_1334974467_477a76e92afda2ddc15779b301189d0c.jpg temp/1738927929_3603119_1334974465_ac0129f29c90c88acc40150b283c4c25.jpg temp/1738927929_3603119_1334974226_7acc04d6e177bdeea66438f46725ca57.jpg temp/1738927929_3603119_1334974214_7a1e529f3a2d5c42936786517ca86c3e.jpg temp/1738927929_3603119_1334974210_ff53bb856fe8f2b9c055af3404e33719.jpg temp/1738927929_3603119_1334974208_cfcb01373b350a2445668206f99bfa5c.jpg temp/1738927929_3603119_1334974163_bbb5ac2741b50c6b9d426bc7dbfcfcd7.jpg temp/1738927929_3603119_1334974155_48295a6c8ccb30431c96a91c3c64a4cd.jpg temp/1738927929_3603119_1334973669_f216f24617bd1b128e5ddd4c710e1d14.jpg temp/1738927929_3603119_1334973645_16b41197a5e84fcf96da577ef170e969.jpg temp/1738927929_3603119_1334973602_fb0c01fc3dd49549d83aaa00c3462f43.jpg temp/1738927929_3603119_1334973547_0f180f3b4122c9f8c1c2571b27ccdfdb.jpg temp/1738927929_3603119_1334973497_c230f76652b1175e634913b3956c3614.jpg temp/1738927929_3603119_1334973489_66071e5c05c5f791144358e0c268a4c9.jpg temp/1738927929_3603119_1334973175_93c834b97a741ac727edba29a9cf7d08.jpg temp/1738927929_3603119_1334973169_a040ad090ed6cdd53f8e45a3525bb506.jpg temp/1738927929_3603119_1334973160_8f78f31e76d55b719be487d4ac08a314.jpg temp/1738927929_3603119_1334973158_6521420f8ea4ef415df6436a918db35f.jpg temp/1738927929_3603119_1334973156_c886d6c0b1149d05d8d8dffa9ac25464.jpg temp/1738927929_3603119_1334973151_720aff388eef8f45e242bea81b346354.jpg temp/1738927929_3603119_1334973095_c8d97ec27bd7f5276b5a0e4f9f0b5814.jpg temp/1738927929_3603119_1334973082_b9d5050ab1cde1fccddb22ab641b8ec7.jpg temp/1738927929_3603119_1334973079_e4377c4f9d505c03b90731d4818097f3.jpg temp/1738927929_3603119_1334973077_b9bd69101704ce34b7c74ba5ffb234a2.jpg temp/1738927929_3603119_1334973075_a6ba7426a93abc6d4bfae783abc85e45.jpg temp/1738927929_3603119_1334973074_1eafb68f8d9646173a9def19c1d2b05d.jpg temp/1738927929_3603119_1334973072_838e11b70be1566153403c48b5627dab.jpg temp/1738927929_3603119_1334973070_14fdce4f756e258b4682a5bd285bdf1e.jpg temp/1738927929_3603119_1334973065_4c29661b51c4bed7214b221c2afef24f.jpg temp/1738927929_3603119_1334973060_9d94b74f35dd6bb6f52c47de1f341608.jpg temp/1738927929_3603119_1334973056_d8089de487037aa041cd9b64a7696500.jpg temp/1738927929_3603119_1334973054_6979324a73c0ba0b407ce74b79ea84aa.jpg temp/1738927929_3603119_1334972989_85dcc73c578d9e3943419714f686aaae.jpg temp/1738927929_3603119_1334972905_3e357b1746b76344e1f69779e79a1339.jpg temp/1738927929_3603119_1334972903_d439187778334bd3c88f19435f8fa30b.jpg temp/1738927929_3603119_1334972900_bcd40c421362213fbba1da10339c08b7.jpg temp/1738927929_3603119_1334972896_13cdfd457419384533b99fe2d4eb63ad.jpg temp/1738927929_3603119_1334972891_24359d1f4f57ded2ade93ddee2d9881e.jpg temp/1738927929_3603119_1334972714_55a75e4dcb9567c5afde5900cc779799.jpg temp/1738927929_3603119_1334972713_667111b070e9e992818f3e30cd2ec7c5.jpg temp/1738927929_3603119_1334972706_f36a4d2de73523348fca9d6cd24ee088.jpg temp/1738927929_3603119_1334972612_5dfe0bd7d190345149a91ed13b41e17d.jpg temp/1738927929_3603119_1334972515_5c9bdfc67141859069e8c5440702888b.jpg temp/1738927929_3603119_1334972418_a572654da1417324545978e3b7a4948a.jpg temp/1738927929_3603119_1334972341_39e428439b33f5c887c7875216fddc25.jpg temp/1738927929_3603119_1334972339_ac3ae905d23b95df0b16813b3e9ae99c.jpg temp/1738927929_3603119_1334972337_0ce43ac330a6a513d078a4b28cd52e42.jpg temp/1738927929_3603119_1334972334_dedcc5e8b033bf59d96a7bc48c90072a.jpg temp/1738927929_3603119_1334972332_568299de9a85f64c71937d3f55887179.jpg temp/1738927929_3603119_1334972331_481ef977f591911aaa2b9803b839484d.jpg temp/1738927929_3603119_1334972304_7300eccab1492fbb6cc4d29ca007f02b.jpg temp/1738927929_3603119_1334972298_1ac84beacd97ae163c572b85a09561c5.jpg temp/1738927929_3603119_1334972292_51fa66db61636436dcc1f8db98c9971d.jpg temp/1738927929_3603119_1334972289_3f4503dd988a16f4aa5ddab7cf86ba61.jpg temp/1738927929_3603119_1334972286_c28554a194111d957abab17041ba0088.jpg temp/1738927929_3603119_1334972284_e59d6e8a36f75380574c517bffd7991e.jpg Found 300 images belonging to 1 classes. begin to do the prediction : ERROR in datou_step_exec, will save and exit ! Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above. [[{{node model/module/StatefulPartitionedCall/StatefulPartitionedCall/StatefulPartitionedCall/conv1/StatefulPartitionedCall/Conv2D}}]] [Op:__inference_predict_function_34490] Function call stack: predict_function File "/home/admin/workarea/git/Velours/python/mtr/datou/datou_lib.py", line 2329, in datou_exec output = datou_step_exec(sNext, args, cache, context, map_info, verbose, mtr_user_id) File "/home/admin/workarea/git/Velours/python/mtr/datou/datou_lib.py", line 2523, in datou_step_exec return lib_process.datou_step_tfhub2(param, json_param, args, cache, context, map_info, verbose) File "/home/admin/workarea/git/Velours/python/mtr/datou/lib_step_exec/lib_step_process.py", line 3146, in datou_step_tfhub2 classes, outputs, features = this_model.predict_image_paths(list_paths, keep_aspect_ratio=keep_aspect_ratio, File "/home/admin/workarea/git/Velours/python/mtr/tfhub2/evaluate.py", line 288, in predict_image_paths Y_pred, F_pred = self.model.predict(valid_generator, validation_steps) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/training.py", line 88, in _method_wrapper return method(self, *args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/training.py", line 1268, in predict tmp_batch_outputs = predict_function(iterator) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py", line 580, in __call__ result = self._call(*args, **kwds) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py", line 650, in _call return self._concrete_stateful_fn._filtered_call(canon_args, canon_kwds) # pylint: disable=protected-access File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 1661, in _filtered_call return self._call_flat( File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 1745, in _call_flat return self._build_call_outputs(self._inference_function.call( File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 593, in call outputs = execute.execute( File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/execute.py", line 59, in quick_execute tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name, [1334982433, 1334982429, 1334982426, 1334982424, 1334982423, 1334982421, 1334982392, 1334982391, 1334982390, 1334982387, 1334982384, 1334982376, 1334982100, 1334982099, 1334982095, 1334982087, 1334982065, 1334982062, 1334982022, 1334981955, 1334981918, 1334981915, 1334981908, 1334981906, 1334981276, 1334981205, 1334981142, 1334981077, 1334981062, 1334981057, 1334981030, 1334981027, 1334981024, 1334981021, 1334981018, 1334981014, 1334980964, 1334980961, 1334980957, 1334980953, 1334980939, 1334980930, 1334980878, 1334980835, 1334980826, 1334980825, 1334980824, 1334980822, 1334980766, 1334980762, 1334980759, 1334980756, 1334980753, 1334980750, 1334980522, 1334980454, 1334980366, 1334980249, 1334980227, 1334980221, 1334980170, 1334980167, 1334980165, 1334980161, 1334980157, 1334980152, 1334980097, 1334980095, 1334980083, 1334980081, 1334980077, 1334980075, 1334980066, 1334980042, 1334980030, 1334980028, 1334980024, 1334980023, 1334979907, 1334979902, 1334979897, 1334979892, 1334979888, 1334979885, 1334979849, 1334979845, 1334979840, 1334979835, 1334979830, 1334979826, 1334979576, 1334979572, 1334979569, 1334979565, 1334979562, 1334979557, 1334979356, 1334979352, 1334979348, 1334979344, 1334979342, 1334979340, 1334979307, 1334979300, 1334979296, 1334979291, 1334979287, 1334979285, 1334979194, 1334979187, 1334979170, 1334979168, 1334979151, 1334979149, 1334979100, 1334979097, 1334979096, 1334979094, 1334979061, 1334979058, 1334978992, 1334978989, 1334978974, 1334978934, 1334978929, 1334978784, 1334978624, 1334978622, 1334978621, 1334978617, 1334978615, 1334978607, 1334978365, 1334978361, 1334978219, 1334978176, 1334978138, 1334978105, 1334977633, 1334977448, 1334977363, 1334977317, 1334977315, 1334977312, 1334976798, 1334976797, 1334976795, 1334976792, 1334976770, 1334976765, 1334976715, 1334976712, 1334976708, 1334976707, 1334976697, 1334976696, 1334976684, 1334976683, 1334976680, 1334976678, 1334976677, 1334976675, 1334976650, 1334976647, 1334976646, 1334976630, 1334976558, 1334976503, 1334976478, 1334976473, 1334976471, 1334976468, 1334976466, 1334976460, 1334976393, 1334976390, 1334976370, 1334976328, 1334976324, 1334976273, 1334976153, 1334976147, 1334976145, 1334976144, 1334976143, 1334976141, 1334976120, 1334976118, 1334976117, 1334976116, 1334976114, 1334976111, 1334975999, 1334975976, 1334975974, 1334975961, 1334975959, 1334975941, 1334975517, 1334975378, 1334975375, 1334975374, 1334975371, 1334975348, 1334975320, 1334975318, 1334975316, 1334975314, 1334975312, 1334975311, 1334975267, 1334975262, 1334975260, 1334975258, 1334975256, 1334975254, 1334975242, 1334975240, 1334975238, 1334975235, 1334975232, 1334975229, 1334975209, 1334975206, 1334975056, 1334975045, 1334975043, 1334975041, 1334974977, 1334974973, 1334974972, 1334974963, 1334974906, 1334974902, 1334974716, 1334974714, 1334974712, 1334974710, 1334974707, 1334974705, 1334974657, 1334974645, 1334974549, 1334974469, 1334974467, 1334974465, 1334974226, 1334974214, 1334974210, 1334974208, 1334974163, 1334974155, 1334973669, 1334973645, 1334973602, 1334973547, 1334973497, 1334973489, 1334973175, 1334973169, 1334973160, 1334973158, 1334973156, 1334973151, 1334973095, 1334973082, 1334973079, 1334973077, 1334973075, 1334973074, 1334973072, 1334973070, 1334973065, 1334973060, 1334973056, 1334973054, 1334972989, 1334972905, 1334972903, 1334972900, 1334972896, 1334972891, 1334972714, 1334972713, 1334972706, 1334972612, 1334972515, 1334972418, 1334972341, 1334972339, 1334972337, 1334972334, 1334972332, 1334972331, 1334972304, 1334972298, 1334972292, 1334972289, 1334972286, 1334972284] begin to insert list_values into mtr_datou_result : length of list_values in save_final : 300 time used for this insertion : 0.44579124450683594 save_final ERROR in last step tfhub_classification2, Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above. [[{{node model/module/StatefulPartitionedCall/StatefulPartitionedCall/StatefulPartitionedCall/conv1/StatefulPartitionedCall/Conv2D}}]] [Op:__inference_predict_function_34490] Function call stack: predict_function time spend for datou_step_exec : 26.021459817886353 time spend to save output : 0.465808629989624 total time spend for step 0 : 26.487268447875977 need to delete datou_research and reload, so keep current state 1 caffe_path_current : About to save ! 2 After save, about to update current ! 15.54user 8.49system 1:35.43elapsed 25%CPU (0avgtext+0avgdata 1563228maxresident)k 2953568inputs+1131136outputs (16365major+792377minor)pagefaults 0swaps