ReadMeFeed / Community knowledge
Real questions. Useful conversations. Find the people who know your stack.
Enable mixed‑precision in Keras in one line, understand compute vs variable dtypes, and avoid common pitfalls like late policy setting and missing float32 output layers.
Mixed‑precision training can boost speed and reduce memory but often triggers NaNs, accuracy drops, or runtime errors. This guide walks through a diagnostic flow, checks, and fixes to get reliable FP16 training on TensorFlow 2.x.
Learn how to apply PyTorch's automatic mixed precision to cut training time and memory usage, with a ready‑to‑use code pattern and practical verification steps.
Learn when to use PyTorch AMP, the minimal correct integration, trust boundaries around GradScaler, operational checks, and the failure modes that could force you to stop using mixed precision.
The four-line AMP training-loop change, the float16 vs bfloat16 decision, and the trade-offs that decide whether mixed precision actually speeds up your PyTorch training.
When Keras mixed‑precision training stalls with NaN loss, the culprit is often hidden in unsupported layers, missing loss scaling, or incompatible optimizers. This diagnostic guide walks through a concise checklist, quick checks, and concrete fixes to get your model back on track.
Keras mixed precision is a one-line change that can meaningfully speed up GPU training — but only with the right hardware, a float32 output layer, and loss scaling. Here's how to enable it and verify the win.
Enable mixed_float16 in Keras, keep the output layer in float32, handle loss scaling correctly for custom loops, and verify against a float32 baseline with clear recovery options.
I'm setting up automatic mixed precision training with torch.autocast and torch.cuda.amp.GradScaler and want to make a deliberate choice between two documented configurations: float16 autocast with gradient scaling, or bfloat16 autocast without it. My understanding is that bfloat16's wider exponent range usually makes the scaler unnecessary, while float16 ne
keras
tensorflow
pytorch