Keras Callbacks (Model_CheckPoint,Early_Stopping,ReduceLROnPlateau)

Keras

by DeepLearning Engineer 2022. 1. 1. 20:26

SMALL

Keras_Callbacks

Keras Callbacks (Model_CheckPoint,Early_Stopping,ReduceLROnPlateau)¶

안녕하세요. 오늘은 Keras에서 좋은 모델을 얻기위해 하는 기법중 3개를 소개시켜 드리려고합니다. 모두 모델을 좀더 잘 튜닝하기위해서 사용하는것들이며, 사용법이 비슷하기때문에 한번에 소개해 드리려고해요.

1. Model_CheckPoint¶

model checkpoint 는 주기적으로 모델을 저장할때 사용하는 객체입니다.
이 옵션을 잘 사용하면 나중에 가장 성능이 좋은모델을 사용할 수 있습니다.
옵션 :
filepath = "저장경로"
verbose=1 /모델에서 변경사항이 생겼을 경우 알려주는 옵션이라고 생각하시면 됩니다.
monitor = "loss","val_loss","accuracy","val_accuracy" 등등...
save_best_only =True 일 경우 monitor에서 더 좋은 지표가 나왔을 경우에 저장
save_weights_only = True 일 경우 weights만 저장

위와같은 옵션이 대표적으로 존재하며 더 많은 옵션을 보시려면 케라스 공식 홈페이지를 참고하시면 됩니다.
https://keras.io/api/callbacks/model_checkpoint/

2. Early_Stopping¶

early stopping 은 monitor 하는 지표의 값의 개선이 없을때 (지정 횟수 이상) 모델의 훈련을 멈추게 합니다.

옵션:
monitor = "loss","val_loss","accuracy","val_accuracy" ...
verbose=1 /모델에서 변경사항이 생겼을 경우 알려주는 옵션이라고 생각하시면 됩니다.
patience = 이후 훈련이 중지되는 개선 사항이 없는 Epoch 수입니다.

위와같은 옵션이 대표적으로 존재하며 더 많은 옵션을 보시려면 케라스 공식 홈페이지를 참고하시면 됩니다.
https://keras.io/api/callbacks/early_stopping/

3. ReduceLROnPlateau¶

메트릭 개선이 중지된 경우 학습 속도를 줄입니다.

모델들은 학습이 정체되면 학습률을 2-10배 감소시킴으로써 종종 이익을 얻는다. 이 콜백은 양을 모니터링하고 '인내성' 횟수에 대한 개선이 보이지 않으면 학습 속도가 감소합니다.

옵션:
factor = 학습률이 감소되는 정도 new_lr = lr*factor
verbose=1 /모델에서 변경사항이 생겼을 경우 알려주는 옵션이라고 생각하시면 됩니다.
monitor = "loss","val_loss","accuracy","val_accuracy" ...
patience = 이후 훈련이 중지되는 개선 사항이 없는 Epoch 수입니다.

위의 3개의 옵션을 모두 사용해서 mnist를 다시 훈련시켜 보겠습니다.

In [1]:

from tensorflow.keras.datasets import mnist
import numpy as np
from tensorflow.keras.utils import to_categorical
from tensorflow.keras.models import Sequential,load_model
from tensorflow.keras.layers import Dense,Flatten,BatchNormalization,Conv2D
from tensorflow.keras.optimizers import Adam
import matplotlib.pyplot as plt

(x_train,y_train),(x_test,y_test) = mnist.load_data()
x_train = x_train.reshape(-1,28,28,1)/255.
x_test = x_test.reshape(-1,28,28,1)/255.
y_train = to_categorical(y_train)
y_test = to_categorical(y_test)

model = Sequential()
model.add(Conv2D(32,(2,2),activation="relu",input_shape=(28,28,1)))
model.add(BatchNormalization())
model.add(Conv2D(64,(2,2),activation="relu"))
model.add(BatchNormalization())
model.add(Conv2D(128,(2,2),2,activation="relu"))
model.add(BatchNormalization())
model.add(Conv2D(32,(2,2),activation="relu"))
model.add(BatchNormalization())
model.add(Conv2D(64,(2,2),activation="relu"))
model.add(BatchNormalization())
model.add(Conv2D(128,(2,2),2,activation="relu"))
model.add(BatchNormalization())
model.add(Flatten())
model.add(Dense(128,activation="relu"))
model.add(Dense(10,activation="softmax"))

Mnist 데이터를 불러온 후 모델까지 정의하였습니다.

In [2]:

from tensorflow.keras.callbacks import ModelCheckpoint,EarlyStopping,ReduceLROnPlateau
checkpointer = ModelCheckpoint(filepath="./models/model.h5",save_best_only=True,verbose=1)
early_stopping = EarlyStopping(patience=10,monitor="val_loss",verbose=1)
reduce_lr = ReduceLROnPlateau(factor = 0.8,patience = 5 , monitor = "val_loss",verbose=1)
optimizer = Adam(learning_rate = 0.001)
model.compile(loss = "categorical_crossentropy",optimizer = optimizer,metrics=["acc"])
history = model.fit(x_train,y_train,validation_data=(x_test,y_test),epochs=50,batch_size=256,callbacks=[checkpointer,early_stopping,reduce_lr])

Epoch 1/50
235/235 [==============================] - 5s 10ms/step - loss: 0.1324 - acc: 0.9596 - val_loss: 4.1170 - val_acc: 0.2406

Epoch 00001: val_loss improved from inf to 4.11697, saving model to ./models\model.h5
Epoch 2/50
235/235 [==============================] - 2s 9ms/step - loss: 0.0300 - acc: 0.9907 - val_loss: 0.1738 - val_acc: 0.9501

Epoch 00002: val_loss improved from 4.11697 to 0.17376, saving model to ./models\model.h5
Epoch 3/50
235/235 [==============================] - 2s 9ms/step - loss: 0.0192 - acc: 0.9933 - val_loss: 0.0406 - val_acc: 0.9886

Epoch 00003: val_loss improved from 0.17376 to 0.04058, saving model to ./models\model.h5
Epoch 4/50
235/235 [==============================] - 2s 9ms/step - loss: 0.0141 - acc: 0.9955 - val_loss: 0.0333 - val_acc: 0.9909

Epoch 00004: val_loss improved from 0.04058 to 0.03325, saving model to ./models\model.h5
Epoch 5/50
235/235 [==============================] - 2s 9ms/step - loss: 0.0104 - acc: 0.9965 - val_loss: 0.0421 - val_acc: 0.9876

Epoch 00005: val_loss did not improve from 0.03325
Epoch 6/50
235/235 [==============================] - 2s 9ms/step - loss: 0.0066 - acc: 0.9977 - val_loss: 0.0392 - val_acc: 0.9905

Epoch 00006: val_loss did not improve from 0.03325
Epoch 7/50
235/235 [==============================] - 2s 9ms/step - loss: 0.0076 - acc: 0.9970 - val_loss: 0.0546 - val_acc: 0.9874

Epoch 00007: val_loss did not improve from 0.03325
Epoch 8/50
235/235 [==============================] - 2s 9ms/step - loss: 0.0128 - acc: 0.9962 - val_loss: 0.0545 - val_acc: 0.9876

Epoch 00008: val_loss did not improve from 0.03325
Epoch 9/50
235/235 [==============================] - 2s 9ms/step - loss: 0.0103 - acc: 0.9966 - val_loss: 0.0466 - val_acc: 0.9902

Epoch 00009: val_loss did not improve from 0.03325

Epoch 00009: ReduceLROnPlateau reducing learning rate to 0.000800000037997961.
Epoch 10/50
235/235 [==============================] - 2s 9ms/step - loss: 0.0049 - acc: 0.9984 - val_loss: 0.0423 - val_acc: 0.9906

Epoch 00010: val_loss did not improve from 0.03325
Epoch 11/50
235/235 [==============================] - 2s 9ms/step - loss: 0.0020 - acc: 0.9994 - val_loss: 0.0365 - val_acc: 0.9920

Epoch 00011: val_loss did not improve from 0.03325
Epoch 12/50
235/235 [==============================] - 2s 9ms/step - loss: 0.0011 - acc: 0.9997 - val_loss: 0.0346 - val_acc: 0.9928

Epoch 00012: val_loss did not improve from 0.03325
Epoch 13/50
235/235 [==============================] - 2s 9ms/step - loss: 2.8471e-04 - acc: 0.9999 - val_loss: 0.0322 - val_acc: 0.9928

Epoch 00013: val_loss improved from 0.03325 to 0.03220, saving model to ./models\model.h5
Epoch 14/50
235/235 [==============================] - 2s 10ms/step - loss: 4.9388e-04 - acc: 0.9999 - val_loss: 0.0318 - val_acc: 0.9925

Epoch 00014: val_loss improved from 0.03220 to 0.03179, saving model to ./models\model.h5
Epoch 15/50
235/235 [==============================] - 2s 9ms/step - loss: 8.6859e-05 - acc: 1.0000 - val_loss: 0.0314 - val_acc: 0.9929

Epoch 00015: val_loss improved from 0.03179 to 0.03137, saving model to ./models\model.h5
Epoch 16/50
235/235 [==============================] - 2s 9ms/step - loss: 2.7105e-05 - acc: 1.0000 - val_loss: 0.0310 - val_acc: 0.9927

Epoch 00016: val_loss improved from 0.03137 to 0.03103, saving model to ./models\model.h5
Epoch 17/50
235/235 [==============================] - 2s 9ms/step - loss: 1.6741e-05 - acc: 1.0000 - val_loss: 0.0310 - val_acc: 0.9929

Epoch 00017: val_loss improved from 0.03103 to 0.03099, saving model to ./models\model.h5
Epoch 18/50
235/235 [==============================] - 2s 9ms/step - loss: 1.4156e-05 - acc: 1.0000 - val_loss: 0.0311 - val_acc: 0.9928

Epoch 00018: val_loss did not improve from 0.03099
Epoch 19/50
235/235 [==============================] - 2s 9ms/step - loss: 1.2623e-05 - acc: 1.0000 - val_loss: 0.0311 - val_acc: 0.9930

Epoch 00019: val_loss did not improve from 0.03099
Epoch 20/50
235/235 [==============================] - 2s 9ms/step - loss: 1.0336e-05 - acc: 1.0000 - val_loss: 0.0310 - val_acc: 0.9930

Epoch 00020: val_loss improved from 0.03099 to 0.03097, saving model to ./models\model.h5
Epoch 21/50
235/235 [==============================] - 2s 9ms/step - loss: 9.0676e-06 - acc: 1.0000 - val_loss: 0.0310 - val_acc: 0.9930

Epoch 00021: val_loss did not improve from 0.03097

Epoch 00021: ReduceLROnPlateau reducing learning rate to 0.0006400000303983689.
Epoch 22/50
235/235 [==============================] - 2s 9ms/step - loss: 7.9568e-06 - acc: 1.0000 - val_loss: 0.0312 - val_acc: 0.9930

Epoch 00022: val_loss did not improve from 0.03097
Epoch 23/50
235/235 [==============================] - 2s 9ms/step - loss: 7.3136e-06 - acc: 1.0000 - val_loss: 0.0312 - val_acc: 0.9930

Epoch 00023: val_loss did not improve from 0.03097
Epoch 24/50
235/235 [==============================] - 2s 9ms/step - loss: 6.7491e-06 - acc: 1.0000 - val_loss: 0.0313 - val_acc: 0.9931

Epoch 00024: val_loss did not improve from 0.03097
Epoch 25/50
235/235 [==============================] - 2s 9ms/step - loss: 6.1559e-06 - acc: 1.0000 - val_loss: 0.0314 - val_acc: 0.9930

Epoch 00025: val_loss did not improve from 0.03097
Epoch 26/50
235/235 [==============================] - 2s 9ms/step - loss: 5.6206e-06 - acc: 1.0000 - val_loss: 0.0315 - val_acc: 0.9931

Epoch 00026: val_loss did not improve from 0.03097

Epoch 00026: ReduceLROnPlateau reducing learning rate to 0.0005120000336319208.
Epoch 27/50
235/235 [==============================] - 2s 9ms/step - loss: 5.1814e-06 - acc: 1.0000 - val_loss: 0.0316 - val_acc: 0.9933

Epoch 00027: val_loss did not improve from 0.03097
Epoch 28/50
235/235 [==============================] - 2s 9ms/step - loss: 4.6686e-06 - acc: 1.0000 - val_loss: 0.0316 - val_acc: 0.9933

Epoch 00028: val_loss did not improve from 0.03097
Epoch 29/50
235/235 [==============================] - 2s 9ms/step - loss: 4.4266e-06 - acc: 1.0000 - val_loss: 0.0317 - val_acc: 0.9933

Epoch 00029: val_loss did not improve from 0.03097
Epoch 30/50
235/235 [==============================] - 2s 9ms/step - loss: 4.0080e-06 - acc: 1.0000 - val_loss: 0.0318 - val_acc: 0.9932

Epoch 00030: val_loss did not improve from 0.03097
Epoch 00030: early stopping

이제 fit 코드를 돌리면 훈련이 진행되는데 저같은 경우에는 epochs 를 50을 줘서 훈련을 했습니다.
위의 훈련 진행사항을 살펴보면,checkpoint 에서 모델이 저장될 때마다 epoch뒤에 알려줍니다.

reduce_lr 을 통해서 learning_rate 가 줄어드는 메세지를 확인할 수 있습니다.

Early_stopping 을 통해서 patience만큼 모델성능의 개선이 없자 훈련이 종료되는것을 확인할 수 있습니다.

In [3]:

loss = history.history["loss"]
acc = history.history["acc"]
val_loss = history.history["val_loss"]
val_acc = history.history["val_acc"]
plt.subplot(1,2,1)
plt.plot(range(len(loss)),loss,label = "Train Loss")
plt.plot(range(len(val_loss)),val_loss,label = "Validation Loss")
plt.grid()
plt.legend()
plt.subplot(1,2,2)
plt.plot(range(len(acc)),acc,label = "Train Accuracy")
plt.plot(range(len(val_acc)),val_acc,label = "Validation Accuracy")
plt.grid()
plt.legend()
plt.show()

마지막으로 위의 과정들을 통해서 만든 가장 좋은 모델을 통해서 인퍼런스 하는 과정입니다. best_model이라는 객체에 저장 된 모델을 넣어서 predict해주는 코드입니다.

In [ ]:

import random
best_model = load_model("./models/model.h5")

In [12]:

index =random.randint(0,9999)
plt.imshow(x_test[index],cmap="gray")
predict = best_model.predict(x_test[index].reshape(1,28,28,1))
print("Actual : {}\tPredict : {}".format(np.argmax(y_test[index]),np.argmax(predict)),)

Actual : 9	Predict : 9

In [20]:

y_predict = np.argmax(best_model.predict(x_test),axis=1)
y_test = np.argmax(y_test,axis=1)

In [24]:

for index,(true,predict) in enumerate(zip(y_test,y_predict)):
    if true!=predict:
        plt.imshow(x_test[index],cmap="gray")
        print("Actual : {}\tPredict : {}".format(true,predict))
        plt.show()

Actual : 2	Predict : 7

Actual : 5	Predict : 3

Actual : 6	Predict : 0

Actual : 8	Predict : 2

Actual : 2	Predict : 1

Actual : 5	Predict : 3

Actual : 9	Predict : 7

Actual : 8	Predict : 9

Actual : 6	Predict : 5

Actual : 7	Predict : 1

Actual : 7	Predict : 2

Actual : 9	Predict : 4

Actual : 4	Predict : 9

Actual : 9	Predict : 5

Actual : 5	Predict : 3

Actual : 9	Predict : 7

Actual : 8	Predict : 7

Actual : 0	Predict : 6

Actual : 3	Predict : 7

Actual : 8	Predict : 3

Actual : 9	Predict : 4

Actual : 5	Predict : 3

Actual : 4	Predict : 9

Actual : 6	Predict : 1

Actual : 3	Predict : 5

Actual : 9	Predict : 4

Actual : 9	Predict : 4

Actual : 2	Predict : 0

Actual : 2	Predict : 4

Actual : 5	Predict : 3

Actual : 6	Predict : 1

Actual : 9	Predict : 4

Actual : 9	Predict : 7

Actual : 8	Predict : 0

Actual : 9	Predict : 5

Actual : 3	Predict : 5

Actual : 9	Predict : 7

Actual : 7	Predict : 9

Actual : 6	Predict : 0

Actual : 6	Predict : 4

Actual : 4	Predict : 8

Actual : 5	Predict : 0

Actual : 4	Predict : 6

Actual : 4	Predict : 6

Actual : 0	Predict : 2

Actual : 9	Predict : 8

Actual : 2	Predict : 7

Actual : 1	Predict : 7

Actual : 5	Predict : 3

Actual : 2	Predict : 7

Actual : 1	Predict : 9

Actual : 6	Predict : 0

Actual : 6	Predict : 1

Actual : 3	Predict : 5

Actual : 9	Predict : 8

Actual : 4	Predict : 9

Actual : 9	Predict : 4

Actual : 1	Predict : 4

Actual : 3	Predict : 8

Actual : 7	Predict : 1

Actual : 0	Predict : 7

Actual : 8	Predict : 4

Actual : 2	Predict : 8

Actual : 4	Predict : 9

Actual : 7	Predict : 2

Actual : 6	Predict : 1

Actual : 5	Predict : 6

Actual : 5	Predict : 0

Actual : 4	Predict : 9

Actual : 2	Predict : 7

In [ ]:

LIST

'Keras' 카테고리의 다른 글

CIFAR_10 (0)	2021.12.19
Convolution Neural Network(CNN) (0)	2021.12.03
Keras FCN Network (0)	2021.11.28
Keras MNIST 예제코드 (0)	2021.11.20

파이썬 딥러닝

고정 헤더 영역

메뉴 레이어

메뉴 리스트

검색 레이어

검색 영역

상세 컨텐츠

본문 제목

본문

Keras Callbacks (Model_CheckPoint,Early_Stopping,ReduceLROnPlateau)¶

1. Model_CheckPoint¶

2. Early_Stopping¶

3. ReduceLROnPlateau¶

'Keras' 카테고리의 다른 글

관련글 더보기

댓글 영역

추가 정보

티스토리툴바