Diagnosing and Fixing Mixed‑Precision Training Issues in Keras

Problem Overview

Enabling mixed_float16 in Keras can deliver faster training and lower memory usage, but it also introduces subtle pitfalls:

• Loss or gradient values become NaN/Inf after a few iterations.
• Model accuracy stalls or degrades relative to full‑precision runs.
• Runtime errors such as dtype not supported or tf.float16 not supported.
• No speedup or even slower training times.
• Unexpected memory spikes or out‑of‑memory failures.

This guide presents a diagnostic flow to pinpoint the root cause, apply the appropriate fix, and know when to seek further help.

Step 1 – Verify Hardware & TensorFlow Build

Mixed‑precision relies on NVIDIA Tensor Cores, available on GPUs with compute capability 7.0 or higher (e.g., RTX 20/30 series). Older GPUs will silently fall back to float32 or raise errors.

# Run in a Python shell or notebook
import tensorflow as tf
print("TensorFlow version:", tf.__version__)
print("GPU available:", tf.test.is_gpu_available())
print("Physical GPUs:", tf.config.list_physical_devices('GPU'))
# Check compute capability (if TF 2.10+)
for gpu in tf.config.list_physical_devices('GPU'):
    info = tf.config.experimental.get_device_details(gpu)
    print(gpu.name, "compute capability:", info.get('compute_capability'))

Expected: At least one GPU with compute capability ≥7.0. If not, mixed‑precision will not work; consider upgrading hardware or using a cloud GPU instance.

Step 2 – Enable Mixed‑Precision Policy

Set the global policy before building any layers. The policy must be applied to the entire model to avoid dtype mismatches.

from tensorflow.keras import mixed_precision
mixed_precision.set_global_policy('mixed_float16')
# Build model after policy is set
model = tf.keras.Sequential([
    tf.keras.layers.Dense(64, activation='relu'),
    tf.keras.layers.Dense(10)
])
model.summary()

Check that model.summary() shows dtype: float16 for trainable weights and dtype: float32 for the final layer (biases remain float32).

Step 3 – Confirm Loss Scaling

TensorFlow automatically applies loss scaling when the policy is mixed_float16. However, custom training loops or older code may bypass it, leading to NaNs. Verify that loss scaling is active:

policy = mixed_precision.global_policy()
print("Loss scale:", policy.loss_scale)

Expected: loss_scale is a DynamicLossScale instance with a numeric value (e.g., 128). If it’s None, you need to enable it explicitly:

mixed_precision.set_global_policy('mixed_float16')
policy = mixed_precision.global_policy()
policy.loss_scale = mixed_precision.LossScaleOptimizer(tf.keras.optimizers.Adam(), loss_scale=128)

Tip: If you’re using a custom training loop, wrap the optimizer with LossScaleOptimizer and ensure the loss is scaled before back‑propagation.

Step 4 – Detect NaNs/Inf Early

Insert a tf.debugging.check_numerics callback to catch numerical issues during training. This stops training immediately when a NaN or Inf appears.

class CheckNumericsCallback(tf.keras.callbacks.Callback):
    def on_batch_end(self, batch, logs=None):
        tf.debugging.check_numerics(self.model.optimizer.lr, "Learning Rate")
        tf.debugging.check_numerics(self.model.optimizer, "Optimizer")

model.fit(train_ds, epochs=5, callbacks=[CheckNumericsCallback()])

If the callback raises an error, the problem is almost certainly due to loss scaling or dtype mismatches in custom layers.

Step 5 – Verify Performance Gains

Measure training time with and without mixed precision. Use time.time() or Python’s timeit module. Example:

import time
start = time.time()
model.fit(train_ds, epochs=3)
print("Mixed‑precision time:", time.time() - start)

# Disable mixed precision
mixed_precision.set_global_policy('float32')
model_fp32 = tf.keras.models.clone_model(model)
model_fp32.compile(...)
start = time.time()
model_fp32.fit(train_ds, epochs=3)
print("Float32 time:", time.time() - start)

Expected: Mixed‑precision training should be faster (typically 1.5–2×) and use less GPU memory. If the time is similar or slower, the model may not be using FP16 kernels (check tf.config.experimental.get_memory_info('GPU:0') for memory usage).

Step 6 – Inspect Custom Layers and Metrics

Custom layers that perform operations not supported in FP16 (e.g., tf.linalg.matmul with dtype=tf.float32) will cast back to float32, negating benefits and possibly causing mismatches. Ensure all custom layers accept and return float16:

class MyCustomLayer(tf.keras.layers.Layer):
    def __init__(self, units, **kwargs):
        super().__init__(**kwargs)
        self.units = units
    def build(self, input_shape):
        self.w = self.add_weight("kernel", shape=[input_shape[-1], self.units], dtype=self.compute_dtype)
    def call(self, inputs):
        return tf.matmul(inputs, self.w)

Similarly, metrics that compute in float32 may produce inaccurate results. Convert metric computations to float16 or use tf.keras.metrics.MeanSquaredError(dtype='float16').

Step 7 – Check for Out‑of‑Memory or Memory Spikes

Mixed precision reduces memory per tensor, but large batch sizes can still exhaust GPU memory if loss scaling is not applied correctly. Monitor memory before and after training:

info_before = tf.config.experimental.get_memory_info('GPU:0')
# Train model
info_after = tf.config.experimental.get_memory_info('GPU:0')
print("Memory used:", info_after['current'] - info_before['current'], "bytes")

Unexpected spikes may indicate that the loss is not scaled, causing Inf gradients that consume additional memory.

Step 8 – Escalation Criteria

When all checks above pass but you still see:

• Persistent NaNs or Inf after loss scaling.
• Accuracy that never improves beyond a threshold or degrades compared to float32.
• Runtime errors that reference unsupported dtypes even after confirming GPU support.

Escalate by:

1. Consulting TensorFlow GitHub issues or Stack Overflow for similar bugs.
2. Running the model on a known‑good GPU (e.g., RTX 3090) to isolate hardware‑specific issues.
3. Rebuilding the model from scratch with minimal custom code to see if the problem persists.
4. Reaching out to the Keras/TensorFlow community with reproducible code snippets and logs.

Concrete Example – End‑to‑End Diagnostic Flow

# 1. Check GPU
import tensorflow as tf
print(tf.test.is_gpu_available())

# 2. Set mixed precision
from tensorflow.keras import mixed_precision
mixed_precision.set_global_policy('mixed_float16')

# 3. Build simple model
model = tf.keras.Sequential([
    tf.keras.layers.Dense(128, activation='relu'),
    tf.keras.layers.Dense(10)
])
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy')

# 4. Add numerics callback
class NumericsCallback(tf.keras.callbacks.Callback):
    def on_batch_end(self, batch, logs=None):
        tf.debugging.check_numerics(self.model.losses, "Loss")

# 5. Train and time
import time
start = time.time()
model.fit(x_train, y_train, batch_size=256, epochs=3, callbacks=[NumericsCallback()])
print("Time FP16:", time.time() - start)

# 6. Disable mixed precision and compare
mixed_precision.set_global_policy('float32')
model_fp32 = tf.keras.models.clone_model(model)
model_fp32.compile(optimizer='adam', loss='sparse_categorical_crossentropy')
start = time.time()
model_fp32.fit(x_train, y_train, batch_size=256, epochs=3)
print("Time FP32:", time.time() - start)

Review the printed times and any numerics errors. If FP16 shows a clear speedup and no NaNs, the configuration is correct. If not, revisit steps 3–6.

Summary

Mixed‑precision training in Keras can be a powerful optimization, but it requires careful setup:

• Verify GPU support (compute capability ≥7.0).
• Set the global policy before model construction.
• Ensure loss scaling is active and applied to the entire loss.
• Detect NaNs early with tf.debugging.check_numerics.
• Measure performance and memory usage to confirm gains.
• Audit custom layers and metrics for dtype compatibility.
• Escalate only when all diagnostics fail to resolve the issue.

Following this diagnostic flow will help you reliably harness FP16 training and avoid common pitfalls.