First, the advice to install a separate instance is to make sure that have the new environment correctly configured before trying to do the upgrade. So with a fresh db and alf_data, does it run as expected with your specific config and customizations.
It it does, then
1. Use your current db that you want to upgrade (that you correctly so have backed up)
2. Yes make a copy of alf_data (or move if you have a backup in another safe storage location, duplicating can take a lot of space)
Before starting, make sure all your configurations point to the correct db and alf_data.
If you do it this way, you will be safe. But you can actually just replace alfresco.war and share.war (tomcat/webapps) in your current install from the 3.4.d install, then delete alfresco and share dir in tomcat/webapps, and start tomcat. The upgrade should take place.