Stop training early and keep the best Keras weights

The problem: your model trains longer than it should

You start a Keras training run with 100 epochs. After epoch 15, validation loss stops improving, but training loss keeps falling. The model is memorizing the training data, not generalizing. If you let it run to epoch 100, you waste compute and may ship a worse model than the one from epoch 15. Stopping by guesswork is unreliable: stop too early and the model underfits; stop too late and it overfits.

The useful takeaway: pair EarlyStopping with ModelCheckpoint so training halts automatically when validation progress stalls, and the model keeps the weights from its best epoch rather than the last one.

How EarlyStopping decides when to halt

EarlyStopping is a Keras callback that runs at the end of each epoch. You tell it which metric to watch—usually val_loss—and how many epochs to wait without improvement before stopping. That wait period is called patience. When patience expires, training ends. If you set restore_best_weights=True, Keras reverts the model to the weights from the epoch with the best monitored metric.

A common mistake is monitoring loss instead of val_loss. Training loss can keep decreasing while validation performance degrades, so stopping on training loss usually means you keep training past the point of overfitting.

How ModelCheckpoint preserves the best state

ModelCheckpoint saves the model or its weights to disk. With save_best_only=True, it overwrites the file only when the monitored metric improves. This protects you from a late training run that overwrites a good checkpoint with a worse one.

You can save the entire model (save_weights_only=False, default) or just the weights (save_weights_only=True). Whole-model files include architecture and optimizer state; weight-only files are smaller but require you to rebuild the model before loading.

EarlyStopping versus ModelCheckpoint

Callback	Primary purpose	Typical monitor	Effect on training
EarlyStopping	Stop when metric stops improving	val_loss	Ends fit() early
ModelCheckpoint	Save best model or weights to disk	val_loss	Writes files; does not stop training

Worked example: configure both callbacks

Run this in a Python script or Jupyter notebook where the user account has write permission to the checkpoint directory. This assumes Keras 3 or tf.keras in TensorFlow 2.x.

import keras
from keras import layers, models
from keras.callbacks import EarlyStopping, ModelCheckpoint

# Placeholder: replace with your actual data
# (X_train, y_train), (X_val, y_val) = ...

model = models.Sequential([
    layers.Dense(64, activation='relu', input_shape=(20,)),
    layers.Dense(32, activation='relu'),
    layers.Dense(1)
])
model.compile(optimizer='adam', loss='mse')

checkpoint_path = './best_model.keras'

early_stop = EarlyStopping(
    monitor='val_loss',
    patience=7,
    min_delta=0.0001,
    restore_best_weights=True,
    mode='min'
)

checkpoint = ModelCheckpoint(
    filepath=checkpoint_path,
    monitor='val_loss',
    save_best_only=True,
    mode='min'
)

history = model.fit(
    X_train, y_train,
    validation_data=(X_val, y_val),
    epochs=100,
    callbacks=[early_stop, checkpoint],
    verbose=1
)

Expected checks: the console should show an early-stopping message once validation loss fails to improve for seven consecutive epochs, and ./best_model.keras should exist afterward. A risk is disk I/O overhead if the model is very large and you checkpoint every epoch; save_best_only=True limits writes to improvements only.

Limitations and how to verify

• Patience too low: a small patience value can stop training before the model converges. Start with 5 to 10 and adjust based on your validation curve.
• Wrong metric: monitoring training loss instead of validation loss typically leads to overfitting.
• Restore scope: restore_best_weights only applies to EarlyStopping. ModelCheckpoint saves to disk but does not restore anything automatically.

Practical verification: after training, load the saved file and evaluate it on the validation set. Compare that loss to the final epoch's validation loss in history.history. If the saved model performs better, the callbacks worked as intended.

Closing

Do not rely on the final epoch. Set monitor='val_loss', patience high enough to avoid premature stops, restore_best_weights=True, and save_best_only=True. Then let Keras decide when training is done.