Chris> We have used lrep repeatedly to push out a snapmirror copy of Chris> our tools repository (2T and growing) to 8 sites so far and Chris> more on the way. Not using it for SV but the premise is the Chris> same.
Sounds like you're running into the same issue as I am, which is that SV sucks over the WAN, but works just dandy over the LAN.
Chris> We break it up and then compress and write the files to 400G PC Chris> SATA drives. We ship the drives to the new location, copy and Chris> uncompress the files back up to a contiguous area and then use Chris> the writer to rebuild it. We have not shipped those files Chris> across the WAN.... would take too long. We are generally out Chris> of sync for 3 weeks by the time it is all said and done and the Chris> resync takes us several days. Still, it is better than the Chris> estimated months of time it was going to take us to do this Chris> across the WAN.
I'm close to three weeks out of sync now (672+ hours) and I'm just not going to be happy when the next updates takes 24+ hours to bring upto speed. It's just pathetic.
Chris> As I understand it you cannot use the lrep r/w more than once Chris> unless they have come up with new tools. Also the lrep cannot Chris> die in the middle or you have to start it over (unlike the Chris> restart of an init of a snapmirror).
I wish they would let me lrep_ multiple times from the master down to the various sub-snapshots. Then I could push the data across faster.
Chris> As to the figuring out how much data you have to move, that is Chris> the manual snap delta calculations as far as I know.
Unfortunately, there doesn't seem to be any sort of reality between the snapvault status -l report and the snapvault delta report, since the first is on a qtree basis and the second is only a volume basis. And the delta report seems to be off by a couple of orders of magnitude, unless it's really showing Mb of difference, not Kb.
Dunno... it's frustrating.
Also, someone else wrote me telling me to use a WAN accelerator, such as those by Riverbed. I messed up in my initial post. We already *are* using WAN accelerators. Helps, but not alot. I can still use 'bbcp' to burn all my WAN bandwidth, while SV won't come anywhere close, accelerator or not.
And I know if I do the same SV across the LAN to the R200 next to my FAS960, I could do it much much much faster, which tells me it's all NetApp's fault due to lack of TCP tuning.
If they'd only let me up the TCP window size to something like 4mb on each end, I'd hope to get better performance. Or something.
Thanks, John
Chris> John Stoffel wrote:
Hi all,
Does anyone else here do SnapVault across the WAN across a long wide link (dual T3, 90+ msec delay) and get decent performance, esp for the initial copy of the volume(s) and/or qtree(s)?
We've gone the route where I did an lrep_reader dump of a 3.8tb qtree (don't ask...) to local disks. Then I used a nice tool called 'bbcp' to push it all across the WAN so that I could actually *use* all my bandwidth.
Regular SnapVault sucks for performance, it just can't push more than 15mb/s which stinks when you have dual T-3 (90mb/s) bandwidth. Using bbcp I can fill that pipe for days on end.
So then once it's all across the WAN, I then used lrep_writer to dump the data to it's destination qtree. All fine and good, but then I have to start a regular snapvault to now catchup with all the data written during the 8+ days the first stage took. Sigh...
Anyone know if I can lrep_reader/lrep_writer the next snapvault snapshot across as well, so I can push the data faster? Because each time the link goes down between the sites, I have to start all over again with the catchup snapvault transfer and it's killing me.
Also, I'd love to be able to *know* how much data is left to copy from the source to the destination, but "snapvault status -l ...." doesn't give that sort of information. Or do I need to start using the 'snap delta' command and my own math to figure out things?
Thanks, John John Stoffel - Senior Staff Systems Administrator - System LSI Group Toshiba America Electronic Components, Inc. - http://www.toshiba.com/taec john.stoffel@taec.toshiba.com - 508-486-1087
Chris> --
Chris> ----------------------------------------------------------------------------- Chris> * Chris Blackmor _______ | There's only three things * Chris> * Advanced Micro Devices ____ | | thats for sure..... * Chris> * Phone: (512) 602-1608 /| | | | taxes, death, and trouble * Chris> * Fax: (512) 602-5155 | |___| | | Rickie Lee Jones * Chris> * Email: chris.blackmor@amd.com |____/ | | "Trouble Man" * Chris> ----------------------------------------------------------------------------- Chris> * My comments are mine, and mine alone. * Chris> -----------------------------------------------------------------------------