Background:
The robocopies were from a "regular" virtual server and virtual drive to NAS. We were migrating to NAS with DDS and the NAS shares are not accessible except from the other OnBase servers via the NAS proxy user. We could not do the copies during business hours since we had to use the live OnBase servers to do the copying.
We have one very large disk group composed of over 3000 volumes of about 1.5 TB among the 68 disk groups copied. We had over 3 TB at the time.It took 5 weekday nights and 1 full weekend to copy over 3 TB and to copy that 3 TB to a second drive. The bandwidth available was around 200-300 Mbs sustained.
- I promoted the current volumes of all of our disk groups to the destination platter. Subsequent volumes are created on the new disk thereafter.
- I leveraged OnBase's sequentially numbered folder-naming scheme to create individual robocopy commands for each volume of each disk group. I used Excel to create the commands so that I could just enter the disk group name and beginning volume number and the spreadsheet created the individual robocopy commands that I then exported to CMD file whose name was the disk group and the volume range it was copying. The log file for each CMD file was the same name as the CMD, but with a different extension of course.
- I discovered that Robocopy copied the volumes much faster this way. I was able to have as many as 25 robocopies running simultaneously, each copying different, mutually exclusive volume ranges.
- If using the /RH: switch, Robocopy won't stop in the middle of copy. The Volume-by-Volume copy gave it the "opportunity" pause during business hours, which it could not have done if it had started copying a complete, large disk group just before the ending time specifed in the switch.
2a. I also created a Copy 2 at the same time, interleaving a robocopy commands that copied the just-copied Copy 1 volume to Copy 2.
3. After each week-day night of copying, before start of business, I called our Hyland Team and ask them to do update the relevant range of volumes.
Because of the amount of time it took, there was a chance that some sort of network, server or storage malfunction might happen. It did. The destination drive filled up unexpectedly and the Copy 2 NAS crashed, but the Robocopy logs told me which volumes had not copied and the spreadsheet let me create ammended CMD files to continue from the last successfully-copied Volume.
I started with some test copies to get an idea of how many volumes I could copy in an hour. I monitored the CPU usage to see how many simultaneous robocopies I could get away with.
4. I ran disk analyses to double-check that robocopy hadn't missed anything (or that i hadn't missed something in its logs) with special attention to those volumes whose copying had been interrupted by the network and hardware problems.
Here's a sample of the interleaved robocopying i did.
robocopy \\<Source SHARE>\<Disk Group>\V10 \\<Copy 1 SHARE>\<Disk Group>\V10 /E /Z /NFL /LOG+:X:\Logs\Copy1-<Disk Group>.243-1.txt /TEE /R:2 /W:10 /RH:1900-0500
robocopy \\<Copy 1 SHARE>\<Disk Group>\V10 \\<Copy 2 SHARE>\<Disk Group>\V10 /E /Z /NFL /LOG+:X:\Logs\Copy2-<Disk Group>.243-1.txt /TEE /R:2 /W:10 /RH:1900-0500
/S /E are mutually exclusive. I used /E because I wanted any empty folders to be copied. I used /NFL because I only wanted confirmation that the folders were copying: the Disk Analysis will let you know if individual files don't copy. I used /TEE so that I could monitor the progress in real time. The /RH: was the weekday time interval during which I wanted the copies to occur.
