Which GradScaler settings matter when choosing float16 over bfloat16 for AMP training?

I'm setting up automatic mixed precision training with torch.autocast and torch.cuda.amp.GradScaler and want to make a deliberate choice between two documented configurations: float16 autocast with gradient scaling, or bfloat16 autocast without it. My understanding is that bfloat16's wider exponent range usually makes the scaler unnecessary, while float16 needs scaling to avoid gradient underflow.

What I can't resolve from the documentation alone is how much the scaler's tuning knobs actually matter. GradScaler exposes init_scale, growth_interval, and backoff_factor, and it dynamically raises the scale after successful iterations and backs off when infs or NaNs appear in gradients. The defaults presumably work for common cases, but it's unclear when a model (e.g., one with very small gradients or loss spikes) justifies changing them, and what symptoms indicate the defaults are hurting convergence rather than helping.

Assume a recent PyTorch release on an Ampere-class GPU; I know the AMP API signatures have shifted across versions, so answers should note version assumptions.

Specifically:

• Is there a documented or empirically grounded criterion for picking init_scale, or is it purely trial and error per model?
• When both dtypes are available on the hardware, is there any reason to prefer float16 plus scaling over bfloat16 besides kernel coverage?
• Does a frequently backing-off scaler (scale dropping repeatedly) indicate a tuning problem or a genuine numerical instability in the model?