Guys,
Anyone know if it's possible do an NDMP backup (using CommVault) of a *named* snapshot? Basically I need to snapshot two or more volumes at the same time (I can do this in a pre-script) and then let the snapshots be backed up.
I guess I could be silly and just create a flex clone volumes and then back those volumes up instead, which would get rid of the issue, but would complicate things a bit more.
Thanks, John John Stoffel - Senior Staff Systems Administrator - System LSI Group Toshiba America Electronic Components, Inc. - http://www.toshiba.com/taec john.stoffel@taec.toshiba.com - 508-486-1087
You should be able to specify "/vol/volname/../snapname" in the backup set. I did it with another backup product before. When a snapshot is specified, ONTAP doesn't create a snap and uses the specified snap (and doesn't delete after like the temp snap when none is specified).
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of John Stoffel Sent: Wednesday, May 09, 2012 10:54 AM To: toasters@teaparty.net Subject: NDMP backups of named snapshots
Guys,
Anyone know if it's possible do an NDMP backup (using CommVault) of a *named* snapshot? Basically I need to snapshot two or more volumes at the same time (I can do this in a pre-script) and then let the snapshots be backed up.
I guess I could be silly and just create a flex clone volumes and then back those volumes up instead, which would get rid of the issue, but would complicate things a bit more.
Thanks, John John Stoffel - Senior Staff Systems Administrator - System LSI Group Toshiba America Electronic Components, Inc. - http://www.toshiba.com/taec john.stoffel@taec.toshiba.com - 508-486-1087 _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Even if it is possible, I would recommend against it. Performance of an NDMP backup from a named snapshot is significantly slower than a volume level NDMP backup.
Are you performing your backup directly from your source volume or a snapvault secondary volume?
On Wed, May 9, 2012 at 1:53 PM, John Stoffel john.stoffel@taec.toshiba.comwrote:
Guys,
Anyone know if it's possible do an NDMP backup (using CommVault) of a *named* snapshot? Basically I need to snapshot two or more volumes at the same time (I can do this in a pre-script) and then let the snapshots be backed up.
I guess I could be silly and just create a flex clone volumes and then back those volumes up instead, which would get rid of the issue, but would complicate things a bit more.
Thanks, John John Stoffel - Senior Staff Systems Administrator - System LSI Group Toshiba America Electronic Components, Inc. - http://www.toshiba.com/taec john.stoffel@taec.toshiba.com - 508-486-1087 _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Bill> Even if it is possible, I would recommend against it. Bill> Performance of an NDMP backup from a named snapshot is Bill> significantly slower than a volume level NDMP backup.
Blech, not good to hear.
Bill> Are you performing your backup directly from your source volume Bill> or a snapvault secondary volume?
Ideally, directly from the source volumes, plural. I'm going to need to backup the volumes from a snapshot time that's as close as possible to the same time, then I can take my time doing the actual NDMP to tape of the data.
So it's looking like my best plan would be to just do:
foreach v (data_a data_b data_c) snap create $v flexclone_backup vol clone create ${v}_clone_bkp -s none -b ${v} flexclone_backup done
then backup my new data_a_clone_bkup... volumes via NDMP.
John
Bill> On Wed, May 9, 2012 at 1:53 PM, John Stoffel Bill> john.stoffel@taec.toshiba.comwrote:
Guys,
Anyone know if it's possible do an NDMP backup (using CommVault) of a *named* snapshot? Basically I need to snapshot two or more volumes at the same time (I can do this in a pre-script) and then let the snapshots be backed up.
I guess I could be silly and just create a flex clone volumes and then back those volumes up instead, which would get rid of the issue, but would complicate things a bit more.
Thanks, John John Stoffel - Senior Staff Systems Administrator - System LSI Group Toshiba America Electronic Components, Inc. - http://www.toshiba.com/taec john.stoffel@taec.toshiba.com - 508-486-1087 _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
When you start a volume level NDMP backup, Data ONTAP takes a snapshot, does its backup from that snapshot, then deletes that snapshot. If the data is basically static at the time you wish to perform your backup then you could take your named snapshot for faster recovery purposes and immediately thereafter start the volume level NDMP backup - assuming you have available destination devices. Otherwise, what you have described will likely give the best performance.
On Wed, May 9, 2012 at 2:33 PM, John Stoffel john.stoffel@taec.toshiba.comwrote:
Bill> Even if it is possible, I would recommend against it. Bill> Performance of an NDMP backup from a named snapshot is Bill> significantly slower than a volume level NDMP backup.
Blech, not good to hear.
Bill> Are you performing your backup directly from your source volume Bill> or a snapvault secondary volume?
Ideally, directly from the source volumes, plural. I'm going to need to backup the volumes from a snapshot time that's as close as possible to the same time, then I can take my time doing the actual NDMP to tape of the data.
So it's looking like my best plan would be to just do:
foreach v (data_a data_b data_c) snap create $v flexclone_backup vol clone create ${v}_clone_bkp -s none -b ${v} flexclone_backup done
then backup my new data_a_clone_bkup... volumes via NDMP.
John
Bill> On Wed, May 9, 2012 at 1:53 PM, John Stoffel Bill> john.stoffel@taec.toshiba.comwrote:
Guys,
Anyone know if it's possible do an NDMP backup (using CommVault) of a *named* snapshot? Basically I need to snapshot two or more volumes at the same time (I can do this in a pre-script) and then let the snapshots be backed up.
I guess I could be silly and just create a flex clone volumes and then back those volumes up instead, which would get rid of the issue, but would complicate things a bit more.
Thanks, John John Stoffel - Senior Staff Systems Administrator - System LSI Group Toshiba America Electronic Components, Inc. -
john.stoffel@taec.toshiba.com - 508-486-1087 _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Bill> When you start a volume level NDMP backup, Data ONTAP takes a Bill> snapshot, does its backup from that snapshot, then deletes that Bill> snapshot. If the data is basically static at the time you wish Bill> to perform your backup then you could take your named snapshot Bill> for faster recovery purposes and immediately thereafter start Bill> the volume level NDMP backup - assuming you have available Bill> destination devices.
That's the kicker, I probably *won't* have enough available tape drives to run all the volumes in parallel, since I need to make sure my backups (snapshots) are taken close in time.
Bill> Otherwise, what you have described will likely give the best Bill> performance.
Can you give an idea of show slow NDMP backup of a named snapshot is versus a regular NDMP backup using it's own snapshot? 25% slowdown? 50%? I'm running on a FAS3140 which shouldn't too loaded at all.
I'm also working within the limitations of CommVault 9, so I may not be able to specify a named snapshot to backup from there anyway.
Thanks for all your feedback. John
Bill> On Wed, May 9, 2012 at 2:33 PM, John Stoffel Bill> john.stoffel@taec.toshiba.comwrote:
Bill> Even if it is possible, I would recommend against it. Bill> Performance of an NDMP backup from a named snapshot is Bill> significantly slower than a volume level NDMP backup.
Blech, not good to hear.
Bill> Are you performing your backup directly from your source volume Bill> or a snapvault secondary volume?
Ideally, directly from the source volumes, plural. I'm going to need to backup the volumes from a snapshot time that's as close as possible to the same time, then I can take my time doing the actual NDMP to tape of the data.
So it's looking like my best plan would be to just do:
foreach v (data_a data_b data_c) snap create $v flexclone_backup vol clone create ${v}_clone_bkp -s none -b ${v} flexclone_backup done
then backup my new data_a_clone_bkup... volumes via NDMP.
John
Bill> On Wed, May 9, 2012 at 1:53 PM, John Stoffel Bill> john.stoffel@taec.toshiba.comwrote:
Guys,
Anyone know if it's possible do an NDMP backup (using CommVault) of a *named* snapshot? Basically I need to snapshot two or more volumes at the same time (I can do this in a pre-script) and then let the snapshots be backed up.
I guess I could be silly and just create a flex clone volumes and then back those volumes up instead, which would get rid of the issue, but would complicate things a bit more.
Thanks, John John Stoffel - Senior Staff Systems Administrator - System LSI Group Toshiba America Electronic Components, Inc. -
john.stoffel@taec.toshiba.com - 508-486-1087 _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
With the NAS iData Agent you should be able to drill down to the named snapshot as your data source (e.g. /vol/myvol/.snapshot/mysnapshot). You could recycle the named snapshot to rename or delete the previous snapshot then create a new one with the same name.
snap rename myvol mysnapshot mysnapshot.1 snap create myvol mysnapshot
DOH!!! While I was typing this I remembered that I was thinking of in terms of NDMP backup performance was QTree vs Volume, not Volume vs named snapshot. Your named snapshot is essentially a volume level backup so it should not be signifcantly different in performance. QTree backups are much slower because it has to determine which files belong to the QTree then which blocks those files use.
On Wed, May 9, 2012 at 2:45 PM, John Stoffel john.stoffel@taec.toshiba.comwrote:
Bill> When you start a volume level NDMP backup, Data ONTAP takes a Bill> snapshot, does its backup from that snapshot, then deletes that Bill> snapshot. If the data is basically static at the time you wish Bill> to perform your backup then you could take your named snapshot Bill> for faster recovery purposes and immediately thereafter start Bill> the volume level NDMP backup - assuming you have available Bill> destination devices.
That's the kicker, I probably *won't* have enough available tape drives to run all the volumes in parallel, since I need to make sure my backups (snapshots) are taken close in time.
Bill> Otherwise, what you have described will likely give the best Bill> performance.
Can you give an idea of show slow NDMP backup of a named snapshot is versus a regular NDMP backup using it's own snapshot? 25% slowdown? 50%? I'm running on a FAS3140 which shouldn't too loaded at all.
I'm also working within the limitations of CommVault 9, so I may not be able to specify a named snapshot to backup from there anyway.
Thanks for all your feedback. John
Bill> On Wed, May 9, 2012 at 2:33 PM, John Stoffel Bill> john.stoffel@taec.toshiba.comwrote:
Bill> Even if it is possible, I would recommend against it. Bill> Performance of an NDMP backup from a named snapshot is Bill> significantly slower than a volume level NDMP backup.
Blech, not good to hear.
Bill> Are you performing your backup directly from your source volume Bill> or a snapvault secondary volume?
Ideally, directly from the source volumes, plural. I'm going to need to backup the volumes from a snapshot time that's as close as possible to the same time, then I can take my time doing the actual NDMP to tape of the data.
So it's looking like my best plan would be to just do:
foreach v (data_a data_b data_c) snap create $v flexclone_backup vol clone create ${v}_clone_bkp -s none -b ${v} flexclone_backup done
then backup my new data_a_clone_bkup... volumes via NDMP.
John
Bill> On Wed, May 9, 2012 at 1:53 PM, John Stoffel Bill> john.stoffel@taec.toshiba.comwrote:
Guys,
Anyone know if it's possible do an NDMP backup (using CommVault) of a *named* snapshot? Basically I need to snapshot two or more volumes
at
the same time (I can do this in a pre-script) and then let the snapshots be backed up.
I guess I could be silly and just create a flex clone volumes and
then
back those volumes up instead, which would get rid of the issue, but would complicate things a bit more.
Thanks, John John Stoffel - Senior Staff Systems Administrator - System LSI Group Toshiba America Electronic Components, Inc. -
john.stoffel@taec.toshiba.com - 508-486-1087 _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Bill> With the NAS iData Agent you should be able to drill down to the Bill> named snapshot as your data source Bill> (e.g. /vol/myvol/.snapshot/mysnapshot). You could recycle the Bill> named snapshot to rename or delete the previous snapshot then Bill> create a new one with the same name.
Bill> snap rename myvol mysnapshot mysnapshot.1 Bill> snap create myvol mysnapshot
Bill> DOH!!! While I was typing this I remembered that I was thinking Bill> of in terms of NDMP backup performance was QTree vs Volume, not Bill> Volume vs named snapshot. Your named snapshot is essentially a Bill> volume level backup so it should not be signifcantly different Bill> in performance. QTree backups are much slower because it has to Bill> determine which files belong to the QTree then which blocks Bill> those files use.
Ah... that explains it. I couldn't figure out why it would be a problem using named snapshots... so I decided to test it out. Performance looks to be just fine so far... we'll see how it ends up being.
Thanks for the hints, I've ended up doing the following:
1. create new snapshots for each volume in the set that are to be backed up. I used a consistent name for the snapshots to make life easier.
2. I then setup a new sub-client, added in the volumes I wanted, and drilled down to the named snapshot and added that.
3. I wrote a pair of simple pre/post scripts which run on a CommVault Media Agent. The pre- script just deletes any existing snapshots with the same name, and them re-creates a new one.
The post- script goes through and deletes the named snapshot as well.
Now to test out the process for restores. Part of the issue is that one or more of the volumes will actually be an FC lun that needs to be restored, then mounted somewhere else and data copied out. Not ideal, but should work for what we need.
Thanks, John
For the LUNs you have to make sure the data is quiesced before you take the snapshot or the data inside the LUN may get corrupted during a restore. If a database is hosted on the LUN you have to put the database into hot backup mode. Snapdrive can be very helpful for this as well as it will ensure all the buffers get flushed to disk prior to taking the snapshot.
If you're going to do a file based backup from the LUNs, why not mount a clone of the LUNs, possibly on a different server, and just use the regular File System iData Agent?
On Wed, May 9, 2012 at 4:04 PM, John Stoffel john.stoffel@taec.toshiba.comwrote:
Bill> With the NAS iData Agent you should be able to drill down to the Bill> named snapshot as your data source Bill> (e.g. /vol/myvol/.snapshot/mysnapshot). You could recycle the Bill> named snapshot to rename or delete the previous snapshot then Bill> create a new one with the same name.
Bill> snap rename myvol mysnapshot mysnapshot.1 Bill> snap create myvol mysnapshot
Bill> DOH!!! While I was typing this I remembered that I was thinking Bill> of in terms of NDMP backup performance was QTree vs Volume, not Bill> Volume vs named snapshot. Your named snapshot is essentially a Bill> volume level backup so it should not be signifcantly different Bill> in performance. QTree backups are much slower because it has to Bill> determine which files belong to the QTree then which blocks Bill> those files use.
Ah... that explains it. I couldn't figure out why it would be a problem using named snapshots... so I decided to test it out. Performance looks to be just fine so far... we'll see how it ends up being.
Thanks for the hints, I've ended up doing the following:
- create new snapshots for each volume in the set that are to be
backed up. I used a consistent name for the snapshots to make life easier.
- I then setup a new sub-client, added in the volumes I wanted, and
drilled down to the named snapshot and added that.
- I wrote a pair of simple pre/post scripts which run on a CommVault
Media Agent. The pre- script just deletes any existing snapshots with the same name, and them re-creates a new one.
The post- script goes through and deletes the named snapshot as well.
Now to test out the process for restores. Part of the issue is that one or more of the volumes will actually be an FC lun that needs to be restored, then mounted somewhere else and data copied out. Not ideal, but should work for what we need.
Thanks, John
Bill> For the LUNs you have to make sure the data is quiesced before Bill> you take the snapshot or the data inside the LUN may get Bill> corrupted during a restore.
Yup, ideally. But my Oracle Admin swears that things will be ok. We'll be doing testing of restores to see if we need to quiesce things in the pre- script to make sure things are ok. So far, it's been ok. I know... famous last words!
Bill> If a database is hosted on the LUN you have to put the database Bill> into hot backup mode. Snapdrive can be very helpful for this as Bill> well as it will ensure all the buffers get flushed to disk prior Bill> to taking the snapshot.
We don't have snapdrive at all, the DBs are all on Solaris 10 boxes, mounting FC LUNs or NFS volumes from the netapp. Ideally, I'd like for us to move to the Oracle DBs on NFS volumes as well sometime, since the LUNs are just a pain in the ass to deal with.
Bill> If you're going to do a file based backup from the LUNs, why not Bill> mount a clone of the LUNs, possibly on a different server, and Bill> just use the regular File System iData Agent?
Speed, we have lots of data to backup and we want to make these backups run as quickly as we can. We figure restores are going to be urgent if we ever have to do one, so minimizing restore time is key. We're also (possibly) going to have to make sure these backups (really, the snapshots of the volumes) happen at a good time. We'll see how it goes.
Bill> On Wed, May 9, 2012 at 4:04 PM, John Stoffel Bill> john.stoffel@taec.toshiba.comwrote:
Bill> With the NAS iData Agent you should be able to drill down to the Bill> named snapshot as your data source Bill> (e.g. /vol/myvol/.snapshot/mysnapshot). You could recycle the Bill> named snapshot to rename or delete the previous snapshot then Bill> create a new one with the same name.
Bill> snap rename myvol mysnapshot mysnapshot.1 Bill> snap create myvol mysnapshot
Bill> DOH!!! While I was typing this I remembered that I was thinking Bill> of in terms of NDMP backup performance was QTree vs Volume, not Bill> Volume vs named snapshot. Your named snapshot is essentially a Bill> volume level backup so it should not be signifcantly different Bill> in performance. QTree backups are much slower because it has to Bill> determine which files belong to the QTree then which blocks Bill> those files use.
Ah... that explains it. I couldn't figure out why it would be a problem using named snapshots... so I decided to test it out. Performance looks to be just fine so far... we'll see how it ends up being.
Thanks for the hints, I've ended up doing the following:
- create new snapshots for each volume in the set that are to be
backed up. I used a consistent name for the snapshots to make life easier.
- I then setup a new sub-client, added in the volumes I wanted, and
drilled down to the named snapshot and added that.
- I wrote a pair of simple pre/post scripts which run on a CommVault
Media Agent. The pre- script just deletes any existing snapshots with the same name, and them re-creates a new one.
The post- script goes through and deletes the named snapshot as well.
Now to test out the process for restores. Part of the issue is that one or more of the volumes will actually be an FC lun that needs to be restored, then mounted somewhere else and data copied out. Not ideal, but should work for what we need.
Thanks, John
Bill> Even if it is possible, I would recommend against it. Bill> Performance of an NDMP backup from a named snapshot is Bill> significantly slower than a volume level NDMP backup.
Hi Guys,
This seems odd to me because NDMP creates a snapshot and backs that up, not the live volume. I can't see why backing up a NDMP created snapshot vs. a manually created snapshot would perform differently. Maybe you are thinking of volume dump vs. subtree dump.
If you dump /vol/volname/.snapshot/snapname then NDMP is smart enough to do a volume dump of the snapshot, instead of a subtree dump.
Blech, not good to hear.
Bill> Are you performing your backup directly from your source volume Bill> or a snapvault secondary volume?
Ideally, directly from the source volumes, plural. I'm going to need to backup the volumes from a snapshot time that's as close as possible to the same time, then I can take my time doing the actual NDMP to tape of the data.
So it's looking like my best plan would be to just do:
foreach v (data_a data_b data_c) snap create $v flexclone_backup vol clone create ${v}_clone_bkp -s none -b ${v} flexclone_backup done
then backup my new data_a_clone_bkup... volumes via NDMP.
John
Bill> On Wed, May 9, 2012 at 1:53 PM, John Stoffel Bill> john.stoffel@taec.toshiba.comwrote:
Guys,
Anyone know if it's possible do an NDMP backup (using CommVault) of a *named* snapshot? Basically I need to snapshot two or more volumes at the same time (I can do this in a pre-script) and then let the snapshots be backed up.
I guess I could be silly and just create a flex clone volumes and then back those volumes up instead, which would get rid of the issue, but would complicate things a bit more.
Thanks, John John Stoffel - Senior Staff Systems Administrator - System LSI Group Toshiba America Electronic Components, Inc. - http://www.toshiba.com/taec john.stoffel@taec.toshiba.com - 508-486-1087 _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Steve Losen scl@virginia.edu phone: 434-924-0640
University of Virginia ITC Unix Support