Emulator Snapshots vs. Physical Device Animation Disabling for Deterministic UI Testing

Answer to the Core Questions

1. Consistency Across Host Environments
Snapshots give the most reproducible frame‑timing because they lock the entire OS state—boot, system services, and pre‑installed apps—into a single image that runs identically on any host that can execute the same AVD. Disabling animations on a physical device removes most animation‑driven jitter, but host‑level differences (GPU drivers, CPU scheduling, background services) still introduce measurable variance.

2. Sufficiency of Animation Suppression
Turning off window, transition, and animator scales on a physical device dramatically reduces timing noise for most UI tests, but it cannot match the isolation a frozen emulator provides. Background processes, OTA‑applied changes, and device‑specific power‑management can still affect frame timing, so for tests that depend on strict timing or system‑level stability, snapshots remain the safer choice.

Confirmed Facts (Based on Current Knowledge)

• Snapshots freeze the entire OS state, eliminating boot‑time and app‑load variability.
• Animation suppression removes the bulk of UI animation jitter but leaves other nondeterministic factors untouched.
• Snapshots are host‑independent as long as the same AVD image is used.
• Physical devices are vulnerable to OTA updates that can revert animation settings or alter system timing.
• Both approaches require periodic maintenance: snapshots after major OS or app updates, and re‑applying animation settings after OTA.

Practical Steps for Your Test Suite

1. Set Up the Snapshot Environment
   • Launch a clean AVD with the target Android version.
   • Install your app and any required test dependencies.
   • From the AVD window, choose Snapshot > Save and name it (e.g., deterministic-base).
   • In your CI, use emulator -avd MyAvdName -snapshot deterministic-base to start tests.
2. Disable Animations on Physical Devices
   • Run these commands before each test run:

     adb shell settings put global window_animation_scale 0
     adb shell settings put global transition_animation_scale 0
     adb shell settings put global animator_duration_scale 0

   • Optionally, verify the values with adb shell settings get global window_animation_scale etc.
   • Run your test suite and capture frame‑time logs.
3. Compare Variance
   • Collect frame‑time metrics for a representative set of UI interactions on both environments.
   • Use a statistical test (e.g., standard deviation comparison) to quantify consistency.
   • Document the variance differences; snapshots should show lower variance.

Diagnostic Detail Needed to Fine‑Tune the Recommendation

Could you specify which Android image (API level & build) you are using for the AVD? Knowing the exact OS version helps determine if the snapshot will remain stable across major releases.

Key Takeaway

For absolute frame‑timing determinism across heterogeneous hosts, use an emulator snapshot. If you need a simpler, hardware‑level solution that still yields acceptable consistency for most UI tests, disabling animations on physical devices is a viable trade‑off, provided you regularly re‑apply the settings after OTA updates.