Which GradScaler settings matter when choosing float16 over bfloat16 for AMP training?
I'm setting up automatic mixed precision training with torch.autocast and torch.cuda.amp.GradScaler and want to make a deliberate choice between two documented configurations: float16 autocast with gradient scaling, or bfloat16 autocast without it. My understanding is that bfloat16's wider exponent range usually makes the scaler unnecessary, while float16 ne
pytorch