an old-school Unix trick is to use tar, piped to tar in another directory
e.g.
tar cf - /some/file | (cd /some/file; tar xf -)
This page gives a good overview of tar, scp and rsync for doing what you are talking about. http://www.crucialp.com/resources/tutorials/server-administration/how-to-cop...
On Thu, Oct 17, 2013 at 5:22 PM, Touretsky, Gregory < gregory.touretsky@intel.com> wrote:
**1. **If this is Netapp to netapp you may try ndmcopy ****
**2. **If you need something more vendor agnostic – you may want to tar the data at the source, copy this as a single file to the remote site and untar it there. You may copy every subdirectory separately in parallel – which will also speed-up things. You may also want to tune TCP window to overcome some of the latency issues – you’d have to use something like rsync over hpn-ssh to leverage it****
*From*: Arnold de Leon [mailto:a-toasters@deleons.coma-toasters@deleons.com]
*Sent*: Thursday, October 17, 2013 11:37 PM *To*: toasters toasters@teaparty.net *Subject*: Re: Slow copy of a directory full of files via an NFS client across a WAN
I just tried NFSv2, similar results. Same deadly embrace of request/reply which is death with a link with latency. ****
For this current test case this took over 5 minutes to copy 11MB of data in 600 files.****
For reference****
NFS (WAN 20ms) to NFS (WAN 20ms): 5 minutes****
NFS (WAN 20 ms) to local disk: 50 seconds****
Local disk to NFS (WAN 20ms ): 3 minutes 50 seconds****
Local disk to local disk: 0.285 seconds****
NFS (LAN 1ms) to NFS (LAN 1ms): 5 seconds****
[I see that Peter also added data. This is a bit of throwback for me, I've been around long enough to deal with the fun Usenet spool directories and joys of lots of small files]****
The irony here is all I want is to make a copy of a directory which would be a trivial operation on the filer itself, I just need another reference to the directory. The filer can make the copy when data is modified. I'm wondering if I can do this with the OnTap API (I don't know about the APIs yet). The other possible work around is using a host that is on the same LAN as the filer to do the copies. I just can't believe that there isn't a simple solution to this.****
Thanks everyone for the ideas and information.****
arnold****
On Thu, Oct 17, 2013 at 2:28 PM, Mike Horwath drechsau@gmail.com wrote:*
Do you require NFSv3?
Why not try NFSv2 - that will bypass the multitude of *ATTR calls going on a bit and may be better.
Only testing will tell, of course.****
Intel Israel (74) Limited
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters