Or pax. To make the image copy:

pax -rw -pe <src> <dest>

Make sure to use a relative path for src (e.g. cd /a/b/c; pax -rw -pe d /toaster).

barry

On Fri, 7 Apr 2000, Bruce Sterling Woodcock wrote:

[cp and tar examples deleted]

Of course, these will not give exact copies of the data you want. You best bet is to use unix dump and restore or ndmpcopy. Barring that, you want to use something like cpio.

Bruce

Actually, looking more closely, the cp -a might actually work. I never use it since it's a gnuism and you can't be sure you'll have gnu cp on your machine. In general, those options are not available, so you're left with using -r, which doesn't preserve special files and such.

The tar I'm not sure what's wrong with; I just remember it not working in a particular case in the past.

Both scripts I think suffer from potential file naming problems if those files have a " in them or start with a - and so on.

Bruce

well, first off, you make sure that the gnu cp is in your path first, or you specify the full path to cp. the other thing to ensure that there are no errors, redirect the output (including stderr) to a file then check the files for errors.

the only files you have to watch for are the root level directories that you're copying, cp -a takes care of the wacky file names based on the fact that it reads the filenames within the program (opendir) rather than on the command line.

-corris

On Fri, 7 Apr 2000, Bruce Sterling Woodcock wrote:

Date: Fri, 7 Apr 2000 13:41:10 -0700 From: Bruce Sterling Woodcock sirbruce@ix.netcom.com To: Robert Johannes rjohanne@damango.net Cc: Corris Randall corris@acc.com, "Rainchik, Aleksandr (MED, Non GE)" Aleksandr.Rainchik@amermsx.med.ge.com, toasters@mathworks.com Subject: Re: Huge one-time data transfer from UNIX to NetApp

Actually, looking more closely, the cp -a might actually work. I never use it since it's a gnuism and you can't be sure you'll have gnu cp on your machine. In general, those options are not available, so you're left with using -r, which doesn't preserve special files and such.

The tar I'm not sure what's wrong with; I just remember it not working in a particular case in the past.

Both scripts I think suffer from potential file naming problems if those files have a " in them or start with a - and so on.

Bruce

it won't what?

mkdir /mnt/toaster/data cd /usr/data foreach dir (*) cp -a "$dir" /mnt/toaster/data & end

this wasn't meant to be a "shell script" per say, it's just meant to be typed into your interactive shell. The only files that can't have a " or a - at the beginning are the directories in /usr/data

-corris

On Fri, 7 Apr 2000, Bruce Sterling Woodcock wrote:

Date: Fri, 7 Apr 2000 14:29:53 -0700 From: Bruce Sterling Woodcock sirbruce@ix.netcom.com To: Corris Randall corris@acc.com Cc: Robert Johannes rjohanne@damango.net, "Rainchik, Aleksandr (MED, Non GE)" Aleksandr.Rainchik@amermsx.med.ge.com, toasters@mathworks.com Subject: Re: Huge one-time data transfer from UNIX to NetApp

...

the only files you have to watch for are the root level directories that you're copying, cp -a takes care of the wacky file names based on the fact that it reads the filenames within the program (opendir) rather than on the command line.

Yes, but that shell script won't.

Bruce

Bruce, Excuse my ignorance, but why will these not give exact copies? what changes? When necessary use find piped to cpio to shuffle large amount of data between netapps, which you indicate does not have this same problem, but I'm curious as to what the problem actually is.

Justin Acklin

Bruce Sterling Woodcock wrote:

[cp and tar examples deleted]

Of course, these will not give exact copies of the data you want. You best bet is to use unix dump and restore or ndmpcopy. Barring that, you want to use something like cpio.

Bruce

Aleksandr Rainchik wrote:

...

I have a question: what should be the best way to transfer 20-25Gb of data (lot's of small files) from UNIX to NetApp?

Can I do ufsdump on UNIX, pipe it through rsh to NetApp and do restore there?

The answer is: yes, you can. And despite all the other suggestions, this would be my preferred method, both for performance and for transparency.

My second choice would be ufsdump piped to ufsrestore running on an NFS client.

Some folks have mentioned using tar, find|cpio, and cp. Here are the problems with these methods:

tar -- has a 200 character limit on pathnames. You can't copy an arbitrarily deep directory tree or one with very long filenames.

find|cpio -- Doesn't work for filenames with embedded newlines (yes a newline can appear in a filename.) There is also a limit on pathname length, although more than 200 characters. GNU find has -print0 and GNU cpio has -0 to get around the newline problem.

cp -r -- follows symlinks instead of preserving them. Looks like GNU's cp -a solves this problem. I don't know if cp -a preserves special files, however. I also don't know if cp -a has a pathname length limitation. Unix itself limits pathnames to 1024 characters, so if cp does not take care to avoid this limit (by changing directory and using relative pathnames) then cp is unable to copy arbitrarily deep directory trees.

Note the pathname limit in Unix does not prevent you from creating extremely deep directory trees. It's actually quite simple:

i=0 while [ $i -lt 5000 ] do mkdir x cd x i=` expr $i + 1 ` done

So dump|restore is preferable because (I think) it has no pathname length limitations. It preserves symlinks, and special files, and handles any legal filename.

Of course the other methods only fail in unusual circumstances. You can still use them if you check your source directory for problems first.

Steve Losen scl@virginia.edu phone: 804-924-0640

University of Virginia ITC Unix Support