The scenario is as follows:
Main site with a filer used as a Nearstore System, in particular as a Snapvault destination. A branch site with a smaller filer backing up its data every night with Snapvault to the main site. Both sites are connected over a rather low bandwidth WAN connection. Restoring single files from the main site's Nearstore System to the branch site's filer is no big deal. But what if we have to restore a complete volume in the Terabyte size? Over the WAN connection this would take days to complete. Here comes in the offline transfer scenario like so
1. Make a dump of the volume from the Nearstore System to a portable raid box
2. Hop in the car with the raid box and drive to the branch site (about 100 kilometers away)
3. Restore the volume from the raid box to the branch site filer
The crux is how to dump a volume to the portable raid box fast enough. I haven't found a way to directly connect external storage to a filer so we definitely need to use a computer to connect the raid box to and then dump the data from the filer over the network onto this computer. We expect to more or less saturate a 1 Gbits/s link. This would allow us to carry out a restore of a 1 Terabyte volume within one working day.
So far we have tried
- Copy the files directly over NFS / CIFS -> too slow as expected, especially if there are many files and what if the volume contains LUNs and/or qtrees?
- Using NetApp's LREP tool (offline SnapVault) -> astonishingly too slow, we only achieved about 200 Mbits/s although neither the filer nor the laptop used in the scenario hit any resource limit
- NetApp's dump command to dump the volume into a file on a second volume and then transfer the dump file over NFS onto the laptop and the raid box. On the branch site copy the dump file over NFS to a second volume on the filer and then restore the main volume from the dump file. -> Especially the creation of the dump file was quite slow and the method doubles space requirements on both filers
Out of curiosity, since all the above methods were too slow, I tried the following. I connected a LUN over iSCSI to the laptop used in the scenario and copied the files from the LUN onto the raid box. I easily managed to saturate the 1 Gbits/s link even with many small files residing in the LUN. So basically, the filer, the laptop and the raid box are capable of saturating 1 Gbits/s. What I'm missing is a method to dump a hole volume, including all its sub structures like qtrees and LUNs, in a decent way out of a filer. Does anybody have tried something similar and found a performing solution or did anybody succeed with one of the three above mentions methods and if so, how exactly did your scenario look like?
Regards, Wolfram Strauss
Have you talked to NetApp about what they might recommend?
How much data might you have to restore? What about buying an additional, separate disk shelf for the filer? You could plug it in and create an aggregate with only those spindles, then do a vol copy to that aggregate. It should then be possible to unplug the shelf and sneakernet it over to the remote site, where you can plug it into that filer and vol copy the data back off. This is all in theory -- I would check with NetApp engineers to make sure this works the way you want it to.
-Adam
On Fri, Nov 1, 2013 at 6:07 AM, Strauss Wolfram Wolfram.Strauss@emmi.comwrote:
The scenario is as follows:****
Main site with a filer used as a Nearstore System, in particular as a Snapvault destination. A branch site with a smaller filer backing up its data every night with Snapvault to the main site. Both sites are connected over a rather low bandwidth WAN connection. Restoring single files from the main site’s Nearstore System to the branch site’s filer is no big deal. But what if we have to restore a complete volume in the Terabyte size? Over the WAN connection this would take days to complete. Here comes in the offline transfer scenario like so****
**1. **Make a dump of the volume from the Nearstore System to a portable raid box****
**2. **Hop in the car with the raid box and drive to the branch site (about 100 kilometers away)****
**3. **Restore the volume from the raid box to the branch site filer** **
The crux is how to dump a volume to the portable raid box fast enough. I haven’t found a way to directly connect external storage to a filer so we definitely need to use a computer to connect the raid box to and then dump the data from the filer over the network onto this computer. We expect to more or less saturate a 1 Gbits/s link. This would allow us to carry out a restore of a 1 Terabyte volume within one working day.****
So far we have tried****
**- **Copy the files directly over NFS / CIFS -> too slow as expected, especially if there are many files and what if the volume contains LUNs and/or qtrees?****
**- **Using NetApp’s LREP tool (offline SnapVault) -> astonishingly too slow, we only achieved about 200 Mbits/s although neither the filer nor the laptop used in the scenario hit any resource limit****
**- **NetApp’s dump command to dump the volume into a file on a second volume and then transfer the dump file over NFS onto the laptop and the raid box. On the branch site copy the dump file over NFS to a second volume on the filer and then restore the main volume from the dump file. -> Especially the creation of the dump file was quite slow and the method doubles space requirements on both filers****
Out of curiosity, since all the above methods were too slow, I tried the following. I connected a LUN over iSCSI to the laptop used in the scenario and copied the files from the LUN onto the raid box. I easily managed to saturate the 1 Gbits/s link even with many small files residing in the LUN. So basically, the filer, the laptop and the raid box are capable of saturating 1 Gbits/s. What I’m missing is a method to dump a hole volume, including all its sub structures like qtrees and LUNs, in a decent way out of a filer. Does anybody have tried something similar and found a performing solution or did anybody succeed with one of the three above mentions methods and if so, how exactly did your scenario look like?****
Regards,****
Wolfram Strauss****
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
How about a redhat box with a few 500g SSD and the OSSV client.
From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Strauss Wolfram Sent: Friday, November 01, 2013 6:07 AM To: toasters@teaparty.net Subject: Methods to offline transfer a large amount of data between two filers
The scenario is as follows:
Main site with a filer used as a Nearstore System, in particular as a Snapvault destination. A branch site with a smaller filer backing up its data every night with Snapvault to the main site. Both sites are connected over a rather low bandwidth WAN connection. Restoring single files from the main site's Nearstore System to the branch site's filer is no big deal. But what if we have to restore a complete volume in the Terabyte size? Over the WAN connection this would take days to complete. Here comes in the offline transfer scenario like so
1. Make a dump of the volume from the Nearstore System to a portable raid box
2. Hop in the car with the raid box and drive to the branch site (about 100 kilometers away)
3. Restore the volume from the raid box to the branch site filer
The crux is how to dump a volume to the portable raid box fast enough. I haven't found a way to directly connect external storage to a filer so we definitely need to use a computer to connect the raid box to and then dump the data from the filer over the network onto this computer. We expect to more or less saturate a 1 Gbits/s link. This would allow us to carry out a restore of a 1 Terabyte volume within one working day.
So far we have tried
- Copy the files directly over NFS / CIFS -> too slow as expected, especially if there are many files and what if the volume contains LUNs and/or qtrees?
- Using NetApp's LREP tool (offline SnapVault) -> astonishingly too slow, we only achieved about 200 Mbits/s although neither the filer nor the laptop used in the scenario hit any resource limit
- NetApp's dump command to dump the volume into a file on a second volume and then transfer the dump file over NFS onto the laptop and the raid box. On the branch site copy the dump file over NFS to a second volume on the filer and then restore the main volume from the dump file. -> Especially the creation of the dump file was quite slow and the method doubles space requirements on both filers
Out of curiosity, since all the above methods were too slow, I tried the following. I connected a LUN over iSCSI to the laptop used in the scenario and copied the files from the LUN onto the raid box. I easily managed to saturate the 1 Gbits/s link even with many small files residing in the LUN. So basically, the filer, the laptop and the raid box are capable of saturating 1 Gbits/s. What I'm missing is a method to dump a hole volume, including all its sub structures like qtrees and LUNs, in a decent way out of a filer. Does anybody have tried something similar and found a performing solution or did anybody succeed with one of the three above mentions methods and if so, how exactly did your scenario look like?
Regards, Wolfram Strauss
-
Adam Levin -
Jordan Slingerland -
Strauss Wolfram