On Wed, 24 May 2000, Kelsey Cummings wrote:
What choices to migrate the data do we have? We've considered tar/cpio over nfs but that's going to take some time even if we migrate one service at a time.
I had to do exactly that just a couple of weeks ago, from two F230's with two shelves of 4GB drives (about 35GB of data on each) to two F740's. It helps if you have multiple quota trees and exports, which allows you to move one filesystem at a time, rather than an entire filer at a time.
I have a script that does an initial ndmpcopy from the F230 filesystem to populate the F740. Then the script goes into an endless loop updating the target filesystem with rsync (http://rsync.samba.org/) until I signal it to stop. This is where dividing up the work into filesystems helps a lot, otherwise rsync would have spent hours building up its file tables for an entire filer. The F230's were providing mail spool storage, so I definitely wanted to suspend service for a couple of hours during our maintenance window so I could do a read-only export, and issue a final rsync to ensure both old and new filesystems were exactly identical.
The work was split over two weekends, with some filesystems mounted from the F740's and some from the F230's during the transition. I would have used incremental dump/restore via ndmpcopy, but I don't believe that accounts for files deleted off the source filesystem, whereas rsync does. rsync has a lot of options to play with. The ones I used are:
rsync --archive --delete --exclude ".snapshot/" --links --recursive --stats --verbose
I deliberately left out --update to ensure all files are copied over, not trusting file timestamps on the source.