Hi,
I'm new to managing Filers - I hope you'll put up with a newb question or two while I get on my feet. I've not been left entirely on my own here - but there are two unix guys here to mange a lot of servers where before there were four of us .. and the two guys who had done this job previously are the ones who left for greener pastures. So I've got a mature and reasonably stable infrastructue to work with at least.
My first task is to move a volume from a busy filer to a non-busy filer.
1. The preferred method (for our shop, I'm told) is to use ndmpcopy
I've got shell access to both filers, they can ping each other. I've got a desination volume created.
ndmpcopy [options] source destination
The source and destination specify a hostname (no problem) and aboslute pathname of the directory to be used for the transfer. Does this mean the absolute pathname of the source/destination volume? Or .. something else? If the former how do I find that?
2. Unless there is a better way to do this? I can of course mount both volumes via NFS from a 3rd host and simply move then but (from what I've read) ndmpcoy is a better choice.
Brian Dunbar Plexus brian.dunbar@plexus.com
Brian:
Whether or not it's the best way to do it is based on lots of things. The nice thing about ndmpcopy is that it's filer->filer (no host in the middle) and because it's based on dump and restore keeps all of your metadata in tact. The thing it does not do is maintain snapshots. For that you'd need to use 'vol copy' (or Volume SnapMirror if you have a license for that). NDMPcopy will only copy the data you have specified, so if it's the active filesystem, it will copy the active filesystem (as it existed when you ran the command, it actually will take a snapshot for the duration of the copy), while vol copy moves all of the existing snapshots to the destination. However, vol copy only works on like volume types (trad->trad, flex->flex) and there's no incremental way to move the data with 'vol copy' (snapmirror can do this) while ndmpcopy can cross volume types if that's needed.
But if your shop likes ndmpcopy, that's fine. The paths are look like this. I'm assuming you are on the source filer:
src_filer> ndmpcopy /vol/srcvol dest_filer:/vol/destvol
If you want to do a level 0, then do an incremental later, use the -l flag to specify your level. I believe the command supports level 0-2.
I hope this answers your question. Feel free to follow up.
-- Adam Fox adamfox@netapp.com
________________________________
From: Brian Dunbar [mailto:Brian.Dunbar@plexus.com] Sent: Tuesday, April 10, 2007 4:44 PM To: toasters@mathworks.com Subject: ndmpcopy and the newb
Hi,
I'm new to managing Filers - I hope you'll put up with a newb question or two while I get on my feet. I've not been left entirely on my own here - but there are two unix guys here to mange a lot of servers where before there were four of us .. and the two guys who had done this job previously are the ones who left for greener pastures. So I've got a mature and reasonably stable infrastructue to work with at least.
My first task is to move a volume from a busy filer to a non-busy filer.
1. The preferred method (for our shop, I'm told) is to use ndmpcopy
I've got shell access to both filers, they can ping each other. I've got a desination volume created.
ndmpcopy [options] source destination
The source and destination specify a hostname (no problem) and aboslute pathname of the directory to be used for the transfer. Does this mean the absolute pathname of the source/destination volume? Or .. something else? If the former how do I find that?
2. Unless there is a better way to do this? I can of course mount both volumes via NFS from a 3rd host and simply move then but (from what I've read) ndmpcoy is a better choice.
Brian Dunbar Plexus brian.dunbar@plexus.com
brian, its full path:
ndmpcopy [options] sourcefiler:/path/to/source/dir destfiler:/path/to/dest/dir
-- michael
On Tue, Apr 10, 2007 at 03:44:08PM -0500, Brian Dunbar wrote:
Hi,
I'm new to managing Filers - I hope you'll put up with a newb question or two while I get on my feet. I've not been left entirely on my own here - but there are two unix guys here to mange a lot of servers where before there were four of us .. and the two guys who had done this job previously are the ones who left for greener pastures. So I've got a mature and reasonably stable infrastructue to work with at least.
My first task is to move a volume from a busy filer to a non-busy filer.
- The preferred method (for our shop, I'm told) is to use ndmpcopy
I've got shell access to both filers, they can ping each other. I've got a desination volume created.
ndmpcopy [options] source destination
The source and destination specify a hostname (no problem) and aboslute pathname of the directory to be used for the transfer. Does this mean the absolute pathname of the source/destination volume? Or .. something else? If the former how do I find that?
- Unless there is a better way to do this? I can of course mount both volumes via NFS from a 3rd host and simply move then but (from what I've read) ndmpcoy is a better choice.
Brian Dunbar Plexus brian.dunbar@plexus.com
if you have the license, snapmirror works the best for this kind of operation. :)
-- Daniel Leeds Senior Systems Administrator Edmunds.com
-----Original Message----- From: owner-toasters@mathworks.com on behalf of Michael Galloway Sent: Tue 4/10/2007 2:36 PM To: Brian Dunbar Cc: toasters@mathworks.com Subject: Re: ndmpcopy and the newb
brian, its full path:
ndmpcopy [options] sourcefiler:/path/to/source/dir destfiler:/path/to/dest/dir
-- michael
On Tue, Apr 10, 2007 at 03:44:08PM -0500, Brian Dunbar wrote:
Hi,
I'm new to managing Filers - I hope you'll put up with a newb question or two while I get on my feet. I've not been left entirely on my own here - but there are two unix guys here to mange a lot of servers where before there were four of us .. and the two guys who had done this job previously are the ones who left for greener pastures. So I've got a mature and reasonably stable infrastructue to work with at least.
My first task is to move a volume from a busy filer to a non-busy filer.
- The preferred method (for our shop, I'm told) is to use ndmpcopy
I've got shell access to both filers, they can ping each other. I've got a desination volume created.
ndmpcopy [options] source destination
The source and destination specify a hostname (no problem) and aboslute pathname of the directory to be used for the transfer. Does this mean the absolute pathname of the source/destination volume? Or .. something else? If the former how do I find that?
- Unless there is a better way to do this? I can of course mount both volumes via NFS from a 3rd host and simply move then but (from what I've read) ndmpcoy is a better choice.
Brian Dunbar Plexus brian.dunbar@plexus.com
If you do this a lot, and you are moving a whole volume or a qtree, beg your boss to buy snapmirror. If that doesn't work, beg your sales rep for a snapmirror eval license, then after it works so awesome, beg your boss to buy it again.
ndmpcopy does work fine though, espeically when mixed with rsync to catch things up. It depends on how active the volume is, because you'll need to rsync or do NDMP incrmentals to catch it up before you switch your mount point.
If you do this a lot (I suspect a lot of people do move stuff around often for the same reaons you suggest) and you have the $$ to consider it, you should check out www.acopia.com. They have really neat magic for dynamically moving stuff around. I haven't tested acopia, but it looks really interesting.
Jerry
--- "Leeds, Daniel" dleeds@edmunds.com wrote:
if you have the license, snapmirror works the best for this kind of operation. :)
-- Daniel Leeds Senior Systems Administrator Edmunds.com
-----Original Message----- From: owner-toasters@mathworks.com on behalf of Michael Galloway Sent: Tue 4/10/2007 2:36 PM To: Brian Dunbar Cc: toasters@mathworks.com Subject: Re: ndmpcopy and the newb
brian, its full path:
ndmpcopy [options] sourcefiler:/path/to/source/dir destfiler:/path/to/dest/dir
-- michael
On Tue, Apr 10, 2007 at 03:44:08PM -0500, Brian Dunbar wrote:
Hi,
I'm new to managing Filers - I hope you'll put up
with a newb question or two while I get on my feet. I've not been left entirely on my own here - but there are two unix guys here to mange a lot of servers where before there were four of us .. and the two guys who had done this job previously are the ones who left for greener pastures. So I've got a mature and reasonably stable infrastructue to work with at least.
My first task is to move a volume from a busy
filer to a non-busy filer.
- The preferred method (for our shop, I'm told)
is to use ndmpcopy
I've got shell access to both filers, they can
ping each other. I've got a desination volume created.
ndmpcopy [options] source destination
The source and destination specify a hostname (no
problem) and aboslute pathname of the directory to be used for the transfer. Does this mean the absolute pathname of the source/destination volume? Or .. something else? If the former how do I find that?
- Unless there is a better way to do this? I can
of course mount both volumes via NFS from a 3rd host and simply move then but (from what I've read) ndmpcoy is a better choice.
Brian Dunbar Plexus brian.dunbar@plexus.com
____________________________________________________________________________________ No need to miss a message. Get email on-the-go with Yahoo! Mail for Mobile. Get started. http://mobile.yahoo.com/mail
Ndmpcopy will also automatically create the directory path you specify in the destination if it does not exist. This is a nice feature if you are collapsing multiple dir's or qtrees such that you do not need to manually create the new structure.
Jeff Kennedy QCT Engineering Compute 858-651-6592
-----Original Message----- From: owner-toasters@mathworks.com
[mailto:owner-toasters@mathworks.com]
On Behalf Of Michael Galloway Sent: Tuesday, April 10, 2007 2:37 PM To: Brian Dunbar Cc: toasters@mathworks.com Subject: Re: ndmpcopy and the newb
brian, its full path:
ndmpcopy [options] sourcefiler:/path/to/source/dir destfiler:/path/to/dest/dir
-- michael
On Tue, Apr 10, 2007 at 03:44:08PM -0500, Brian Dunbar wrote:
Hi,
I'm new to managing Filers - I hope you'll put up with a newb
question
or two while I get on my feet. I've not been left entirely on my own
here
- but there are two unix guys here to mange a lot of servers where
before
there were four of us .. and the two guys who had done this job
previously
are the ones who left for greener pastures. So I've got a mature and reasonably stable infrastructue to work with at least.
My first task is to move a volume from a busy filer to a non-busy
filer.
- The preferred method (for our shop, I'm told) is to use ndmpcopy
I've got shell access to both filers, they can ping each other. I've
got
a desination volume created.
ndmpcopy [options] source destination
The source and destination specify a hostname (no problem) and
aboslute
pathname of the directory to be used for the transfer. Does this mean
the
absolute pathname of the source/destination volume? Or .. something
else?
If the former how do I find that?
- Unless there is a better way to do this? I can of course mount
both
volumes via NFS from a 3rd host and simply move then but (from what
I've
read) ndmpcoy is a better choice.
Brian Dunbar Plexus brian.dunbar@plexus.com
ndmpcopy -sa <username>:<password> -da <username>:<password> <sourcefilername>:/vol/<volname> <destinationfilername>:/vol/<volname>
Sa username/pw = the source filer's credentials, da is the destinations filer's credential
The paths are case sensitive, so make sure it's all correct..
So to answer question 1 the absolute path is /vol/<volname> or if you just want to move a folder in a volume it would be /vol/<volname>/<foldername>
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Brian Dunbar Sent: Tuesday, April 10, 2007 1:44 PM To: toasters@mathworks.com Subject: ndmpcopy and the newb
Hi,
I'm new to managing Filers - I hope you'll put up with a newb question or two while I get on my feet. I've not been left entirely on my own here - but there are two unix guys here to mange a lot of servers where before there were four of us .. and the two guys who had done this job previously are the ones who left for greener pastures. So I've got a mature and reasonably stable infrastructue to work with at least.
My first task is to move a volume from a busy filer to a non-busy filer.
1. The preferred method (for our shop, I'm told) is to use ndmpcopy
I've got shell access to both filers, they can ping each other. I've got a desination volume created.
ndmpcopy [options] source destination
The source and destination specify a hostname (no problem) and aboslute pathname of the directory to be used for the transfer. Does this mean the absolute pathname of the source/destination volume? Or .. something else? If the former how do I find that?
2. Unless there is a better way to do this? I can of course mount both volumes via NFS from a 3rd host and simply move then but (from what I've read) ndmpcoy is a better choice.
Brian Dunbar Plexus brian.dunbar@plexus.com
How many files do you have? ndmpcopy can take a looooong time if you are copying millions and millions of files. If you have similar sized volumes snapmirror (as long as the destination is equal or greater then the source) is a good solution, the only caveat is to make sure you have like disks for the snapmirror. If not, you'll have to reallocate the destination after the migration is done.
As Adam said, vol copy is a good and cheaper solution.
I've taken to adding a user with the useradmin command, then using ndmpd passwd command to create a throwaway password for the ndmpcopy to use. Makes me feel better when passing authentication credentials around.
-Blake
On 4/10/07, Langdon, Laughlin T. (Lock) Langdon.Lock@mayo.edu wrote:
ndmpcopy –sa <username>:<password> -da <username>:<password> <sourcefilername>:/vol/<volname> <destinationfilername>:/vol/<volname>
Sa username/pw = the source filer's credentials, da is the destinations filer's credential
The paths are case sensitive, so make sure it's all correct..
So to answer question 1 the absolute path is /vol/<volname> or if you just want to move a folder in a volume it would be /vol/<volname>/<foldername>
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Brian Dunbar Sent: Tuesday, April 10, 2007 1:44 PM To: toasters@mathworks.com Subject: ndmpcopy and the newb
Hi,
I'm new to managing Filers - I hope you'll put up with a newb question or two while I get on my feet. I've not been left entirely on my own here - but there are two unix guys here to mange a lot of servers where before there were four of us .. and the two guys who had done this job previously are the ones who left for greener pastures. So I've got a mature and reasonably stable infrastructue to work with at least.
My first task is to move a volume from a busy filer to a non-busy filer.
- The preferred method (for our shop, I'm told) is to use ndmpcopy
I've got shell access to both filers, they can ping each other. I've got a desination volume created.
ndmpcopy [options] source destination
The source and destination specify a hostname (no problem) and aboslute pathname of the directory to be used for the transfer. Does this mean the absolute pathname of the source/destination volume? Or .. something else? If the former how do I find that?
- Unless there is a better way to do this? I can of course mount both
volumes via NFS from a 3rd host and simply move then but (from what I've read) ndmpcoy is a better choice.
Brian Dunbar Plexus brian.dunbar@plexus.com
We don't do a lot of NDMPcopies. I mostly utilize DFS replication, and or storageX/VFM to do actual data migration. This allows me to move data without impacting anyone, and it can take a month, and no one really cares/notices. :)
I was just showing the syntax for ndmpcopy
-----Original Message----- From: Blake Golliher [mailto:thelastman@gmail.com] Sent: Tuesday, April 10, 2007 7:16 PM To: Langdon, Laughlin T. (Lock) Cc: Brian Dunbar; toasters@mathworks.com Subject: Re: ndmpcopy and the newb
How many files do you have? ndmpcopy can take a looooong time if you are copying millions and millions of files. If you have similar sized volumes snapmirror (as long as the destination is equal or greater then the source) is a good solution, the only caveat is to make sure you have like disks for the snapmirror. If not, you'll have to reallocate the destination after the migration is done.
As Adam said, vol copy is a good and cheaper solution.
I've taken to adding a user with the useradmin command, then using ndmpd passwd command to create a throwaway password for the ndmpcopy to use. Makes me feel better when passing authentication credentials around.
-Blake
On 4/10/07, Langdon, Laughlin T. (Lock) Langdon.Lock@mayo.edu wrote:
ndmpcopy -sa <username>:<password> -da <username>:<password> <sourcefilername>:/vol/<volname> <destinationfilername>:/vol/<volname>
Sa username/pw = the source filer's credentials, da is the
destinations
filer's credential
The paths are case sensitive, so make sure it's all correct..
So to answer question 1 the absolute path is /vol/<volname> or if
you just
want to move a folder in a volume it would be
/vol/<volname>/<foldername>
From: owner-toasters@mathworks.com
[mailto:owner-toasters@mathworks.com] On
Behalf Of Brian Dunbar Sent: Tuesday, April 10, 2007 1:44 PM To: toasters@mathworks.com Subject: ndmpcopy and the newb
Hi,
I'm new to managing Filers - I hope you'll put up with a newb
question or
two while I get on my feet. I've not been left entirely on my own
here -
but there are two unix guys here to mange a lot of servers where
before
there were four of us .. and the two guys who had done this job
previously
are the ones who left for greener pastures. So I've got a mature and reasonably stable infrastructue to work with at least.
My first task is to move a volume from a busy filer to a non-busy
filer.
- The preferred method (for our shop, I'm told) is to use ndmpcopy
I've got shell access to both filers, they can ping each other. I've
got a
desination volume created.
ndmpcopy [options] source destination
The source and destination specify a hostname (no problem) and
aboslute
pathname of the directory to be used for the transfer. Does this mean
the
absolute pathname of the source/destination volume? Or .. something
else?
If the former how do I find that?
- Unless there is a better way to do this? I can of course mount
both
volumes via NFS from a 3rd host and simply move then but (from what
I've
read) ndmpcoy is a better choice.
Brian Dunbar Plexus brian.dunbar@plexus.com
I haven't looked at the doc's yet, just the help syntax, but is there a way to make ndmpcopy use challenge for its authtype? Currently I have to change it to plaintext each time I want to do this but our backup software wants challenge, which I like better anyway.
I am using a Solaris binary for these transfers, not the onboard version.
Thanks.
Jeff Kennedy QCT Engineering Compute 858-651-6592
-----Original Message----- From: owner-toasters@mathworks.com
[mailto:owner-toasters@mathworks.com]
On Behalf Of Langdon, Laughlin T. (Lock) Sent: Wednesday, April 11, 2007 8:41 AM To: Blake Golliher Cc: Brian Dunbar; toasters@mathworks.com Subject: RE: ndmpcopy and the newb
We don't do a lot of NDMPcopies. I mostly utilize DFS replication,
and
or storageX/VFM to do actual data migration. This allows me to move data without impacting anyone, and it can take a month, and no one really cares/notices. :)
I was just showing the syntax for ndmpcopy
-----Original Message----- From: Blake Golliher [mailto:thelastman@gmail.com] Sent: Tuesday, April 10, 2007 7:16 PM To: Langdon, Laughlin T. (Lock) Cc: Brian Dunbar; toasters@mathworks.com Subject: Re: ndmpcopy and the newb
How many files do you have? ndmpcopy can take a looooong time if you are copying millions and millions of files. If you have similar sized volumes snapmirror (as long as the destination is equal or greater then the source) is a good solution, the only caveat is to make sure you have like disks for the snapmirror. If not, you'll have to reallocate the destination after the migration is done.
As Adam said, vol copy is a good and cheaper solution.
I've taken to adding a user with the useradmin command, then using ndmpd passwd command to create a throwaway password for the ndmpcopy to use. Makes me feel better when passing authentication credentials around.
-Blake
On 4/10/07, Langdon, Laughlin T. (Lock) Langdon.Lock@mayo.edu wrote:
ndmpcopy -sa <username>:<password> -da <username>:<password> <sourcefilername>:/vol/<volname> <destinationfilername>:/vol/<volname>
Sa username/pw = the source filer's credentials, da is the
destinations
filer's credential
The paths are case sensitive, so make sure it's all correct..
So to answer question 1 the absolute path is /vol/<volname> or if
you just
want to move a folder in a volume it would be
/vol/<volname>/<foldername>
From: owner-toasters@mathworks.com
[mailto:owner-toasters@mathworks.com] On
Behalf Of Brian Dunbar Sent: Tuesday, April 10, 2007 1:44 PM To: toasters@mathworks.com Subject: ndmpcopy and the newb
Hi,
I'm new to managing Filers - I hope you'll put up with a newb
question or
two while I get on my feet. I've not been left entirely on my own
here -
but there are two unix guys here to mange a lot of servers where
before
there were four of us .. and the two guys who had done this job
previously
are the ones who left for greener pastures. So I've got a mature
and
reasonably stable infrastructue to work with at least.
My first task is to move a volume from a busy filer to a non-busy
filer.
- The preferred method (for our shop, I'm told) is to use ndmpcopy
I've got shell access to both filers, they can ping each other.
I've
got a
desination volume created.
ndmpcopy [options] source destination
The source and destination specify a hostname (no problem) and
aboslute
pathname of the directory to be used for the transfer. Does this
mean
the
absolute pathname of the source/destination volume? Or .. something
else?
If the former how do I find that?
- Unless there is a better way to do this? I can of course mount
both
volumes via NFS from a 3rd host and simply move then but (from what
I've
read) ndmpcoy is a better choice.
Brian Dunbar Plexus brian.dunbar@plexus.com
Thanks for all of your advice.
Snapmirror was suggested and we do have a license for it - it's been used for a few months to move our databases from filer to filer. Before I feel comfortable using it for 'just data" I'd want to practice in our development environment. I'll make that a to-do, then.
I did use ndmpcopy with level 0, 1, and 2 then eliminate the soft link to the old volume. I'll give it another day and then I'll 0 eliminate the data on the source volume.
The only gotcha was that in my finely crafted checklist I neglected to add 'edit the nfs export' so the volume can be mounted as RW. So following the cutover I could not add files to the destination volume. Luckily while it was in production this way for about 10 minutes no one noticed; that volume is used by our BOM system for attachments and while they reference it a great deal on the 3rd shift the folks who make changes are 1st shift guys.
Thanks for all your advice and pointers.
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Brian Dunbar Sent: Tuesday, April 10, 2007 3:44 PM To: toasters@mathworks.com Subject: ndmpcopy and the newb
Hi,
I'm new to managing Filers - I hope you'll put up with a newb question or two while I get on my feet. I've not been left entirely on my own here - but there are two unix guys here to mange a lot of servers where before there were four of us .. and the two guys who had done this job previously are the ones who left for greener pastures. So I've got a mature and reasonably stable infrastructue to work with at least.
My first task is to move a volume from a busy filer to a non-busy filer.
1. The preferred method (for our shop, I'm told) is to use ndmpcopy
I've got shell access to both filers, they can ping each other. I've got a desination volume created.
ndmpcopy [options] source destination
The source and destination specify a hostname (no problem) and aboslute pathname of the directory to be used for the transfer. Does this mean the absolute pathname of the source/destination volume? Or .. something else? If the former how do I find that?
2. Unless there is a better way to do this? I can of course mount both volumes via NFS from a 3rd host and simply move then but (from what I've read) ndmpcoy is a better choice.
Brian Dunbar Plexus brian.dunbar@plexus.com
-
Blake Golliher -
Brian Dunbar -
Fox, Adam -
Jerry -
Kennedy, Jeffrey -
Langdon, Laughlin T. (Lock) -
Leeds, Daniel -
Michael Galloway