Hello I’m curious what other people are using for software to back up volumes to tape? We’ve recently purchased a Dell TL4000 with 2 x LTO7 drives and 24 tapes. We have a four-node cluster running 8.3.2Px and for now we’re directly connecting the TL4000’s A-side FC to node 1 and the B-side to node 3. AFAIK the device itself is set up correctly and all the physical and virtual status LEDs and are happy.
After installing the correct TCF file, ‘sysconfig –t’ run against the directly connected nodes shows the drive as supported (IBM LTO-7 ULT3580) with all sorts of device name permutations (rewind, no rewind, unload, etc). Running ‘storage tape show’ shows two tape drives, one per directly attached node, named st0 and a media changer on node 3 named mc0. The tape drives are offline, the media changer is available.
I’ve played with the various ‘storage tape’ commands but am making no headway. I’m sure the docs are out there, and I’ve looked through the NetApp documentation as well as Googling for a while, but don’t see a way to control the media changer to, for example, tell tape drive 1 to load slot 6, and then another command to dump a volume to it. I understand we want SVM-scoped ndmp. Does anyone have bare bones examples using ndmp or some other local utility? If we need to purchase something like Tivoli or Netbackup or CommVault etc then I guess we will, but we’d prefer free.
Thanks!
-- Ian Ehrenwald Senior Infrastructure Engineer Hachette Book Group, Inc. 1.617.263.1948 / ian.ehrenwald@hbgusa.com
This may contain confidential material. If you are not an intended recipient, please notify the sender, delete immediately, and understand that no disclosure or reliance on the information herein is permitted. Hachette Book Group may monitor email to and from our network.
blockquote, div.yahoo_quoted { margin-left: 0 !important; border-left:1px #715FFA solid !important; padding-left:1ex !important; background-color:white !important; } Tape??
Crazy talk.
#NeverTooMuchSarcasm
Sent from Yahoo Mail for iPhone
On Monday, May 15, 2017, 12:40, Ehrenwald, Ian Ian.Ehrenwald@hbgusa.com wrote:
Hello I’m curious what other people are using for software to back up volumes to tape? We’ve recently purchased a Dell TL4000 with 2 x LTO7 drives and 24 tapes. We have a four-node cluster running 8.3.2Px and for now we’re directly connecting the TL4000’s A-side FC to node 1 and the B-side to node 3. AFAIK the device itself is set up correctly and all the physical and virtual status LEDs and are happy.
After installing the correct TCF file, ‘sysconfig –t’ run against the directly connected nodes shows the drive as supported (IBM LTO-7 ULT3580) with all sorts of device name permutations (rewind, no rewind, unload, etc). Running ‘storage tape show’ shows two tape drives, one per directly attached node, named st0 and a media changer on node 3 named mc0. The tape drives are offline, the media changer is available.
I’ve played with the various ‘storage tape’ commands but am making no headway. I’m sure the docs are out there, and I’ve looked through the NetApp documentation as well as Googling for a while, but don’t see a way to control the media changer to, for example, tell tape drive 1 to load slot 6, and then another command to dump a volume to it. I understand we want SVM-scoped ndmp. Does anyone have bare bones examples using ndmp or some other local utility? If we need to purchase something like Tivoli or Netbackup or CommVault etc then I guess we will, but we’d prefer free.
Thanks!
-- Ian Ehrenwald Senior Infrastructure Engineer Hachette Book Group, Inc. 1.617.263.1948 / ian.ehrenwald@hbgusa.com
This may contain confidential material. If you are not an intended recipient, please notify the sender, delete immediately, and understand that no disclosure or reliance on the information herein is permitted. Hachette Book Group may monitor email to and from our network.
_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Without back up software, I think you can only control a drive, not a robot. Somebody can feel free to correct me if I'm wrong, I've never tried anything else.
On Mon, May 15, 2017 at 3:36 PM Ehrenwald, Ian Ian.Ehrenwald@hbgusa.com wrote:
Hello I’m curious what other people are using for software to back up volumes to tape? We’ve recently purchased a Dell TL4000 with 2 x LTO7 drives and 24 tapes. We have a four-node cluster running 8.3.2Px and for now we’re directly connecting the TL4000’s A-side FC to node 1 and the B-side to node 3. AFAIK the device itself is set up correctly and all the physical and virtual status LEDs and are happy.
After installing the correct TCF file, ‘sysconfig –t’ run against the directly connected nodes shows the drive as supported (IBM LTO-7 ULT3580) with all sorts of device name permutations (rewind, no rewind, unload, etc). Running ‘storage tape show’ shows two tape drives, one per directly attached node, named st0 and a media changer on node 3 named mc0. The tape drives are offline, the media changer is available.
I’ve played with the various ‘storage tape’ commands but am making no headway. I’m sure the docs are out there, and I’ve looked through the NetApp documentation as well as Googling for a while, but don’t see a way to control the media changer to, for example, tell tape drive 1 to load slot 6, and then another command to dump a volume to it. I understand we want SVM-scoped ndmp. Does anyone have bare bones examples using ndmp or some other local utility? If we need to purchase something like Tivoli or Netbackup or CommVault etc then I guess we will, but we’d prefer free.
Thanks!
-- Ian Ehrenwald Senior Infrastructure Engineer Hachette Book Group, Inc. 1.617.263.1948 / ian.ehrenwald@hbgusa.com
This may contain confidential material. If you are not an intended recipient, please notify the sender, delete immediately, and understand that no disclosure or reliance on the information herein is permitted. Hachette Book Group may monitor email to and from our network.
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
We used NDMP and Symantec NetBackup. Was pretty decent setup, but does put a load on your filer.
From: toasters-bounces@teaparty.net on behalf of Basil basilberntsen@gmail.com Date: Monday, May 15, 2017 at 2:24 PM To: "Ehrenwald, Ian" Ian.Ehrenwald@hbgusa.com, "toasters@teaparty.net" toasters@teaparty.net Subject: Re: Backing up to tape?
Without back up software, I think you can only control a drive, not a robot. Somebody can feel free to correct me if I'm wrong, I've never tried anything else.
On Mon, May 15, 2017 at 3:36 PM Ehrenwald, Ian <Ian.Ehrenwald@hbgusa.commailto:Ian.Ehrenwald@hbgusa.com> wrote: Hello I’m curious what other people are using for software to back up volumes to tape? We’ve recently purchased a Dell TL4000 with 2 x LTO7 drives and 24 tapes. We have a four-node cluster running 8.3.2Px and for now we’re directly connecting the TL4000’s A-side FC to node 1 and the B-side to node 3. AFAIK the device itself is set up correctly and all the physical and virtual status LEDs and are happy.
After installing the correct TCF file, ‘sysconfig –t’ run against the directly connected nodes shows the drive as supported (IBM LTO-7 ULT3580) with all sorts of device name permutations (rewind, no rewind, unload, etc). Running ‘storage tape show’ shows two tape drives, one per directly attached node, named st0 and a media changer on node 3 named mc0. The tape drives are offline, the media changer is available.
I’ve played with the various ‘storage tape’ commands but am making no headway. I’m sure the docs are out there, and I’ve looked through the NetApp documentation as well as Googling for a while, but don’t see a way to control the media changer to, for example, tell tape drive 1 to load slot 6, and then another command to dump a volume to it. I understand we want SVM-scoped ndmp. Does anyone have bare bones examples using ndmp or some other local utility? If we need to purchase something like Tivoli or Netbackup or CommVault etc then I guess we will, but we’d prefer free.
Thanks!
-- Ian Ehrenwald Senior Infrastructure Engineer Hachette Book Group, Inc. 1.617.263.1948 / ian.ehrenwald@hbgusa.commailto:ian.ehrenwald@hbgusa.com
This may contain confidential material. If you are not an intended recipient, please notify the sender, delete immediately, and understand that no disclosure or reliance on the information herein is permitted. Hachette Book Group may monitor email to and from our network.
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Yepper. On volumes with lots of files, that ndmp history/catalog pass can be crazy!
I once tried to use ndmp backup on a volume with 500,000+ files. After 8 hours and a huge load (FAS6080) the ndmp gave out and quit. Eight hours of building the index and it didn't finish.
Spelling errors courtesy of Swype for iOS...
Tim McCarthy, Consultant at Presidio _____________________________ From: Klise, Steve <steve.klise@wwt.commailto:steve.klise@wwt.com> Sent: Monday, May 15, 2017 6:00 PM Subject: Re: Backing up to tape? To: <toasters@teaparty.netmailto:toasters@teaparty.net>, Basil <basilberntsen@gmail.commailto:basilberntsen@gmail.com>, Ehrenwald, Ian <ian.ehrenwald@hbgusa.commailto:ian.ehrenwald@hbgusa.com>
We used NDMP and Symantec NetBackup. Was pretty decent setup, but does put a load on your filer.
From: <<toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net> on behalf of Basil <basilberntsen@gmail.commailto:basilberntsen@gmail.com> Date: Monday, May 15, 2017 at 2:24 PM To: "Ehrenwald, Ian" <Ian.Ehrenwald@hbgusa.commailto:Ian.Ehrenwald@hbgusa.com>, "toasters@teaparty.netmailto:toasters@teaparty.net" <toasters@teaparty.netmailto:toasters@teaparty.net> Subject: Re: Backing up to tape?
Without back up software, I think you can only control a drive, not a robot. Somebody can feel free to correct me if I'm wrong, I've never tried anything else.
On Mon, May 15, 2017 at 3:36 PM Ehrenwald, Ian <Ian.Ehrenwald@hbgusa.commailto:Ian.Ehrenwald@hbgusa.com> wrote: Hello I’m curious what other people are using for software to back up volumes to tape? We’ve recently purchased a Dell TL4000 with 2 x LTO7 drives and 24 tapes. We have a four-node cluster running 8.3.2Px and for now we’re directly connecting the TL4000’s A-side FC to node 1 and the B-side to node 3. AFAIK the device itself is set up correctly and all the physical and virtual status LEDs and are happy.
After installing the correct TCF file, ‘sysconfig –t’ run against the directly connected nodes shows the drive as supported (IBM LTO-7 ULT3580) with all sorts of device name permutations (rewind, no rewind, unload, etc). Running ‘storage tape show’ shows two tape drives, one per directly attached node, named st0 and a media changer on node 3 named mc0. The tape drives are offline, the media changer is available.
I’ve played with the various ‘storage tape’ commands but am making no headway. I’m sure the docs are out there, and I’ve looked through the NetApp documentation as well as Googling for a while, but don’t see a way to control the media changer to, for example, tell tape drive 1 to load slot 6, and then another command to dump a volume to it. I understand we want SVM-scoped ndmp. Does anyone have bare bones examples using ndmp or some other local utility? If we need to purchase something like Tivoli or Netbackup or CommVault etc then I guess we will, but we’d prefer free.
Thanks!
-- Ian Ehrenwald Senior Infrastructure Engineer Hachette Book Group, Inc. 1.617.263.1948 / ian.ehrenwald@hbgusa.commailto:ian.ehrenwald@hbgusa.com
This may contain confidential material. If you are not an intended recipient, please notify the sender, delete immediately, and understand that no disclosure or reliance on the information herein is permitted. Hachette Book Group may monitor email to and from our network.
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Tim> Yepper. On volumes with lots of files, that ndmp history/catalog Tim> pass can be crazy!
Hah! We've got a volume with 50 million files and it sucks for NDMP backups. Fulls are actually not terrible... but the index pass just takes forever.
Tim> I once tried to use ndmp backup on a volume with 500,000+ Tim> files. After 8 hours and a huge load (FAS6080) the ndmp gave out Tim> and quit. Eight hours of building the index and it didn't finish.
This is the probably with *any* file level backup. Unfortunately, there's no good solution in terms of price for backing up Netapps that I'm aware of.
You could buy a cheaper pair of heads with lots and lots of cheap(ish) SATA storage, but it's still god-awful expensive. And the snapmirror licenses aren't cheap either.
In my engineering environments, I've been trying to encourage them to only backup what they need, and to work in scratch areas instead, but it's hard to get people to change.
The Netapp is just so reliable that is really does keep data for years and years without problems. In more cases.
Heh.
But some $WORK orgs have off-site requirements, and they're not willing to pay the price for a remote co-lo Netapp and storage and the bandwidth to make backups work reliably and quickly enough. So off to tape it is. Make's JSOX happy too...
Does anyone that is using NDMP for file-dense restores also notice a HUGE gap in time between starting the restore, and data to begin moving...with no observable load/activity during that odd bizzare Phase-0 period?
We can see -hours- of 0% CPU, 0KB to disk (short of CP label updates to them), which challenges us in other ways because it brings average dataset recover speed/times to low double-digit MB/sec numbers no matter how fast tape is. _________________________________Jeff MohlerTech Yahoo, Storage Architect, Principal(831)454-6712 YPAC Gold Member Twitter: @PrincipalYahoo CorpIM: Hipchat & Iris
On Monday, May 15, 2017 5:58 PM, John Stoffel john@stoffel.org wrote:
Tim> Yepper. On volumes with lots of files, that ndmp history/catalog Tim> pass can be crazy!
Hah! We've got a volume with 50 million files and it sucks for NDMP backups. Fulls are actually not terrible... but the index pass just takes forever.
Tim> I once tried to use ndmp backup on a volume with 500,000+ Tim> files. After 8 hours and a huge load (FAS6080) the ndmp gave out Tim> and quit. Eight hours of building the index and it didn't finish.
This is the probably with *any* file level backup. Unfortunately, there's no good solution in terms of price for backing up Netapps that I'm aware of.
You could buy a cheaper pair of heads with lots and lots of cheap(ish) SATA storage, but it's still god-awful expensive. And the snapmirror licenses aren't cheap either.
In my engineering environments, I've been trying to encourage them to only backup what they need, and to work in scratch areas instead, but it's hard to get people to change.
The Netapp is just so reliable that is really does keep data for years and years without problems. In more cases.
Heh.
But some $WORK orgs have off-site requirements, and they're not willing to pay the price for a remote co-lo Netapp and storage and the bandwidth to make backups work reliably and quickly enough. So off to tape it is. Make's JSOX happy too... _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Jeffrey> Does anyone that is using NDMP for file-dense restores also Jeffrey> notice a HUGE gap in time between starting the restore, and Jeffrey> data to begin moving...with no observable load/activity Jeffrey> during that odd bizzare Phase-0 period?
I'm not too surprised actually... the NDMP is really just a wrapper around the dump/restore built into OnTap, though I admit that maybe this has changed with cDOT.
In older 7-mode, it was just a single file under /root/<vol>/ that held the index of file(s) dumped to tape at which level. So I doubt it's been improved all that much since.
Jeffrey> We can see -hours- of 0% CPU, 0KB to disk (short of CP label Jeffrey> updates to them), which challenges us in other ways because Jeffrey> it brings average dataset recover speed/times to low Jeffrey> double-digit MB/sec numbers no matter how fast tape is.
Is this before or after you select the file(s) to restore? Or are you doing a full volume restore? I wonder if it's sorting the files by tape block or something like that before starting the restore.
Full volume restore...a small rush of data comes off tape at the start, and then a walking-dead number of hours of zero indicated activity assembling the idea of what to do with it.
None of this is particularly a surprise, other than yes, it appears a forgotten 'who cares' section of ONTAP with still, a valuable amount of customer demand for it..and Im wondering if anyone else is applying any pressure to pull this out of the early 2000s.
Yes, there are other ways to manage data protection, but as long as this is still offered, it should receive responsible attention and adapt to what was 1 CPU systems, to now well over a dozen cores to get the job done.
Not doing this a different way than NDMP..is less of a reflection on where we are WRT to the year marked on the calendar, but certain scales make it harder to spend significantly (many multiples) more on a solution that in the end..is still a MAYBE you will use it restore proposition.
Change is in the air, and how/why/value/cost/support/attention to issues like this have become top of mind again.
_________________________________Jeff MohlerTech Yahoo, Storage Architect, Principal(831)454-6712 YPAC Gold Member Twitter: @PrincipalYahoo CorpIM: Hipchat & Iris
On Monday, May 15, 2017 6:15 PM, John Stoffel john@stoffel.org wrote:
Jeffrey> Does anyone that is using NDMP for file-dense restores also Jeffrey> notice a HUGE gap in time between starting the restore, and Jeffrey> data to begin moving...with no observable load/activity Jeffrey> during that odd bizzare Phase-0 period?
I'm not too surprised actually... the NDMP is really just a wrapper around the dump/restore built into OnTap, though I admit that maybe this has changed with cDOT.
In older 7-mode, it was just a single file under /root/<vol>/ that held the index of file(s) dumped to tape at which level. So I doubt it's been improved all that much since.
Jeffrey> We can see -hours- of 0% CPU, 0KB to disk (short of CP label Jeffrey> updates to them), which challenges us in other ways because Jeffrey> it brings average dataset recover speed/times to low Jeffrey> double-digit MB/sec numbers no matter how fast tape is.
Is this before or after you select the file(s) to restore? Or are you doing a full volume restore? I wonder if it's sorting the files by tape block or something like that before starting the restore.
Just a note:
NDMP is a backup control protocol. It does not define a data format. We run into a lot of confusion on that point. Since NDMP is a standard, you might think that an NDMP backup on one storage array could be restored via NDMP on a different array. That's not a valid conclusion. It's not impossible that you can do that, but it depends on the tape format.
When NDMP first came out, it used the "dump" format. It was similar to ufsdump, and you could even restore the data on a Solaris box pretty easily, although you'd lose all the permission data. That's probably still doable to some degree, but it's not supported.
The problem with dump is it's a logical backup. ONTAP has to walk the directory tree, and the more files you have, the longer it takes. That's why there's a second format option - NDMP with SMtape format. It's documented under "Understanding SMTape engine for FlexVol volumes" in the ONTAP documentation.
I've never met anyone using it personally, but it's been around for a while. I think TSM was the first to support it. It basically does a snapmirror operation to a tape drive, which means you preserve the efficiency. Since it's not walking the files, it was far faster than NDMP with the dump format.
I'm pretty sure that you still can't do a single file restore with SMtape format. You'd have to restore the whole thing. That may or may not be sufficient. If a customer just wants an offsite copy of everything as a quarterly archive or protection against disaster, that should be a good option.
I don't work with tape backups any longer, so I don't know anyone personally who's used SMtape, but if someone on Toasters has used it please let us know how it worked out.
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of John Stoffel Sent: Tuesday, May 16, 2017 3:15 AM To: Jeffrey Mohler jmohler@yahoo-inc.com Cc: toasters@teaparty.net Subject: Re: NDMP Performance...
Jeffrey> Does anyone that is using NDMP for file-dense restores also Jeffrey> notice a HUGE gap in time between starting the restore, and Jeffrey> data to begin moving...with no observable load/activity during Jeffrey> that odd bizzare Phase-0 period?
I'm not too surprised actually... the NDMP is really just a wrapper around the dump/restore built into OnTap, though I admit that maybe this has changed with cDOT.
In older 7-mode, it was just a single file under /root/<vol>/ that held the index of file(s) dumped to tape at which level. So I doubt it's been improved all that much since.
Jeffrey> We can see -hours- of 0% CPU, 0KB to disk (short of CP label Jeffrey> updates to them), which challenges us in other ways because it Jeffrey> brings average dataset recover speed/times to low double-digit Jeffrey> MB/sec numbers no matter how fast tape is.
Is this before or after you select the file(s) to restore? Or are you doing a full volume restore? I wonder if it's sorting the files by tape block or something like that before starting the restore.
_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
We use SMTape, the only issue is the limited format support Netapp provides, IIRC two major versions...so it's not a long term backup solution because you have to renew/refresh it often.
It's very fast however compared to an end to end NDMP.
Snapdiff is looking interesting too. _________________________________Jeff MohlerTech Yahoo, Storage Architect, Principal(831)454-6712 YPAC Gold Member Twitter: @PrincipalYahoo CorpIM: Hipchat & Iris
On Monday, May 15, 2017 11:42 PM, "Steiner, Jeffrey" Jeffrey.Steiner@netapp.com wrote:
Just a note:
NDMP is a backup control protocol. It does not define a data format. We run into a lot of confusion on that point. Since NDMP is a standard, you might think that an NDMP backup on one storage array could be restored via NDMP on a different array. That's not a valid conclusion. It's not impossible that you can do that, but it depends on the tape format.
When NDMP first came out, it used the "dump" format. It was similar to ufsdump, and you could even restore the data on a Solaris box pretty easily, although you'd lose all the permission data. That's probably still doable to some degree, but it's not supported.
The problem with dump is it's a logical backup. ONTAP has to walk the directory tree, and the more files you have, the longer it takes. That's why there's a second format option - NDMP with SMtape format. It's documented under "Understanding SMTape engine for FlexVol volumes" in the ONTAP documentation.
I've never met anyone using it personally, but it's been around for a while. I think TSM was the first to support it. It basically does a snapmirror operation to a tape drive, which means you preserve the efficiency. Since it's not walking the files, it was far faster than NDMP with the dump format.
I'm pretty sure that you still can't do a single file restore with SMtape format. You'd have to restore the whole thing. That may or may not be sufficient. If a customer just wants an offsite copy of everything as a quarterly archive or protection against disaster, that should be a good option.
I don't work with tape backups any longer, so I don't know anyone personally who's used SMtape, but if someone on Toasters has used it please let us know how it worked out.
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of John Stoffel Sent: Tuesday, May 16, 2017 3:15 AM To: Jeffrey Mohler jmohler@yahoo-inc.com Cc: toasters@teaparty.net Subject: Re: NDMP Performance...
Jeffrey> Does anyone that is using NDMP for file-dense restores also Jeffrey> notice a HUGE gap in time between starting the restore, and Jeffrey> data to begin moving...with no observable load/activity during Jeffrey> that odd bizzare Phase-0 period?
I'm not too surprised actually... the NDMP is really just a wrapper around the dump/restore built into OnTap, though I admit that maybe this has changed with cDOT.
In older 7-mode, it was just a single file under /root/<vol>/ that held the index of file(s) dumped to tape at which level. So I doubt it's been improved all that much since.
Jeffrey> We can see -hours- of 0% CPU, 0KB to disk (short of CP label Jeffrey> updates to them), which challenges us in other ways because it Jeffrey> brings average dataset recover speed/times to low double-digit Jeffrey> MB/sec numbers no matter how fast tape is.
Is this before or after you select the file(s) to restore? Or are you doing a full volume restore? I wonder if it's sorting the files by tape block or something like that before starting the restore.
_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
We killed those backups. After review, we determined if we had to, we could request and reconstitute the data from another source. We ended up just backing up our intellectual property which took minutes and only a few tapes....until we retired an older system. Then we made it the "backup" and snapmirrored the important stuff to it...got rid of tapes!
--tmac
*Tim McCarthy, **Principal Consultant*
*Proud Member of the #NetAppATeam https://twitter.com/NetAppATeam*
*I Blog at TMACsRack https://tmacsrack.wordpress.com/*
On Mon, May 15, 2017 at 8:51 PM, John Stoffel john@stoffel.org wrote:
Tim> Yepper. On volumes with lots of files, that ndmp history/catalog Tim> pass can be crazy!
Hah! We've got a volume with 50 million files and it sucks for NDMP backups. Fulls are actually not terrible... but the index pass just takes forever.
Tim> I once tried to use ndmp backup on a volume with 500,000+ Tim> files. After 8 hours and a huge load (FAS6080) the ndmp gave out Tim> and quit. Eight hours of building the index and it didn't finish.
This is the probably with *any* file level backup. Unfortunately, there's no good solution in terms of price for backing up Netapps that I'm aware of.
You could buy a cheaper pair of heads with lots and lots of cheap(ish) SATA storage, but it's still god-awful expensive. And the snapmirror licenses aren't cheap either.
In my engineering environments, I've been trying to encourage them to only backup what they need, and to work in scratch areas instead, but it's hard to get people to change.
The Netapp is just so reliable that is really does keep data for years and years without problems. In more cases.
Heh.
But some $WORK orgs have off-site requirements, and they're not willing to pay the price for a remote co-lo Netapp and storage and the bandwidth to make backups work reliably and quickly enough. So off to tape it is. Make's JSOX happy too...
I wish we could do that... but we have offsite requirements that we just can't get away from, along with data retention requirements. We just got asked for a restore of email from 2001... which ain't too likely to actually happen. LOL.
tmac> We killed those backups. After review, we determined if we had tmac> to, we could request and reconstitute the data from another tmac> source. We ended up just backing up our intellectual property tmac> which took minutes and only a few tapes....until we retired an tmac> older system. Then we made it the "backup" and snapmirrored the tmac> important stuff to it...got rid of tapes!
tmac> --tmac
tmac> Tim McCarthy, Principal Consultant
tmac> Proud Member of the #NetAppATeam
tmac> I Blog at TMACsRack
tmac> On Mon, May 15, 2017 at 8:51 PM, John Stoffel john@stoffel.org wrote:
Tim> Yepper. On volumes with lots of files, that ndmp history/catalog Tim> pass can be crazy!
tmac> Hah! We've got a volume with 50 million files and it sucks for NDMP tmac> backups. Fulls are actually not terrible... but the index pass just tmac> takes forever.
Tim> I once tried to use ndmp backup on a volume with 500,000+ Tim> files. After 8 hours and a huge load (FAS6080) the ndmp gave out Tim> and quit. Eight hours of building the index and it didn't finish.
tmac> This is the probably with *any* file level backup. Unfortunately, tmac> there's no good solution in terms of price for backing up Netapps that tmac> I'm aware of.
tmac> You could buy a cheaper pair of heads with lots and lots of cheap(ish) tmac> SATA storage, but it's still god-awful expensive. And the snapmirror tmac> licenses aren't cheap either.
tmac> In my engineering environments, I've been trying to encourage them to tmac> only backup what they need, and to work in scratch areas instead, but tmac> it's hard to get people to change.
tmac> The Netapp is just so reliable that is really does keep data for years tmac> and years without problems. In more cases.
tmac> Heh.
tmac> But some $WORK orgs have off-site requirements, and they're not tmac> willing to pay the price for a remote co-lo Netapp and storage and the tmac> bandwidth to make backups work reliably and quickly enough. So off to tmac> tape it is. Make's JSOX happy too...
Ian> I’m curious what other people are using for software to back up Ian> volumes to tape? We’ve recently purchased a Dell TL4000 with 2 x Ian> LTO7 drives and 24 tapes. We have a four-node cluster running Ian> 8.3.2Px and for now we’re directly connecting the TL4000’s A-side Ian> FC to node 1 and the B-side to node 3. AFAIK the device itself Ian> is set up correctly and all the physical and virtual status LEDs Ian> and are happy.
We're in the process of upgrading to 8.3.x from 8.2.3 cDOT just so we can properly use Netbackup with NDMP at the VServer level. Under 8.2, it just doesnt' work reliably. I'm hoping that this upgrade will make it work properly.
In any case, you should make sure that each head on the cluster has access to all the tape drives, otherwise you'll probably have a bunch of your backups running across the Cluster Network to do your dumps. And you *might* not be able to NDMP from all vservers.
Ian> After installing the correct TCF file, ‘sysconfig –t’ run against Ian> the directly connected nodes shows the drive as supported (IBM Ian> LTO-7 ULT3580) with all sorts of device name permutations Ian> (rewind, no rewind, unload, etc). Running ‘storage tape show’ Ian> shows two tape drives, one per directly attached node, named st0 Ian> and a media changer on node 3 named mc0. The tape drives are Ian> offline, the media changer is available.
Ian> I’ve played with the various ‘storage tape’ commands but am Ian> making no headway. I’m sure the docs are out there, and I’ve Ian> looked through the NetApp documentation as well as Googling for a Ian> while, but don’t see a way to control the media changer to, for Ian> example, tell tape drive 1 to load slot 6, and then another Ian> command to dump a volume to it. I understand we want SVM-scoped Ian> ndmp. Does anyone have bare bones examples using ndmp or some Ian> other local utility? If we need to purchase something like Ian> Tivoli or Netbackup or CommVault etc then I guess we will, but Ian> we’d prefer free.
There's no free NDMP (especially three way NDMP where it's initiated by a server, dumped by a node to a local tape drive, and the index of files and tape positions sent back to the server).
You're going to have to bite the bullet.
Personally, I'd go with EMC's Legato Networker (not that I've used it in anger for years and years... but the CLI is/was excellent), then probably Netbackup 7.7.x which is what we use now.
I've also run CommVault 7, 8 & now 9 for quite a number of years and I dislike it alot. It's a pain to use, indexes expire so damn quickly it's not funny, and it's just not fun to use.
But... it's cheaper than the other options.
John
Hi Ian,
I had a student in one of my courses, and she backs everything up to tape via scripted SMTape instead of NDMP. Fast, reliable, cheap. I remember being thoroughly impressed, in addition to female storage admins being a rarity...
Would that be an alternative? I'm not familiar with the details of her solution, though, or if she had a tape changer. But it was not a small environment (Hamburg/Germany waterworks).
Sebastian
On May 15, 2017 9:32 PM, "Ehrenwald, Ian" Ian.Ehrenwald@hbgusa.com wrote:
Hello I’m curious what other people are using for software to back up volumes to tape? We’ve recently purchased a Dell TL4000 with 2 x LTO7 drives and 24 tapes. We have a four-node cluster running 8.3.2Px and for now we’re directly connecting the TL4000’s A-side FC to node 1 and the B-side to node 3. AFAIK the device itself is set up correctly and all the physical and virtual status LEDs and are happy.
After installing the correct TCF file, ‘sysconfig –t’ run against the directly connected nodes shows the drive as supported (IBM LTO-7 ULT3580) with all sorts of device name permutations (rewind, no rewind, unload, etc). Running ‘storage tape show’ shows two tape drives, one per directly attached node, named st0 and a media changer on node 3 named mc0. The tape drives are offline, the media changer is available.
I’ve played with the various ‘storage tape’ commands but am making no headway. I’m sure the docs are out there, and I’ve looked through the NetApp documentation as well as Googling for a while, but don’t see a way to control the media changer to, for example, tell tape drive 1 to load slot 6, and then another command to dump a volume to it. I understand we want SVM-scoped ndmp. Does anyone have bare bones examples using ndmp or some other local utility? If we need to purchase something like Tivoli or Netbackup or CommVault etc then I guess we will, but we’d prefer free.
Thanks!
-- Ian Ehrenwald Senior Infrastructure Engineer Hachette Book Group, Inc. 1.617.263.1948 / ian.ehrenwald@hbgusa.com
This may contain confidential material. If you are not an intended recipient, please notify the sender, delete immediately, and understand that no disclosure or reliance on the information herein is permitted. Hachette Book Group may monitor email to and from our network.
_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Sebastian> I had a student in one of my courses, and she backs Sebastian> everything up to tape via scripted SMTape instead of Sebastian> NDMP. Fast, reliable, cheap. I remember being thoroughly Sebastian> impressed, in addition to female storage admins being a Sebastian> rarity...
Sebastian> Would that be an alternative? I'm not familiar with the Sebastian> details of her solution, though, or if she had a tape Sebastian> changer. But it was not a small environment Sebastian> (Hamburg/Germany waterworks).
I'm certainly going to look into this for my future backups. The problem of course is that to restore a single file you need to restore it all, starting with the base full snapshot I would think.
But certainly tempting.