How can I safely upgrade Blender while preserving the ability to roll back to the previous version?

Upgrading to a newer Blender release while keeping the option to revert to the current version requires preserving the existing installation, user preferences, add‑ons, and project files. The main uncertainty lies in where Blender stores its data and how to duplicate or back up these locations without interfering with the new install.

What is the recommended method to back up the current Blender installation and user data before upgrading? How can I install the new version alongside the old one without overwriting files? If issues arise, what steps ensure a reliable rollback to the prior version?