toasters October 2004

toasters@lists.teaparty.net

66 participants
89 discussions

RE: Backups of Snapshots
by Alan Biren 05 Oct '04

05 Oct '04

Alan, I agree with your recommendations. However, I usually leave keep a couple of snapshots that were used for backs around for even quicker restores, instead of going to tape. I also use a naming pattern similar to NetApp and create a hidden share to the .snapshot directory as each volume has it's own. Ie: rsh filer snp delte dailybackup.1 rsh filer snap rename dailybackup.0 dailybackup.1 rsh filer snap create volname dailbackup.0 backup \\filer\snapshare\dailbackup.0 <file:///\\filer\snapshare\dailbackup.0> Regards, Alan Biren Senior Systems Engineer/Storage Architect AMC Corporation 48 West 37th Street New York, NY 10018 (212) 736-0111 www.amccorp.com -----Original Message----- From: owner-toasters(a)mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Alan.McLachlan(a)didata.com.au Sent: Tuesday, October 05, 2004 12:38 AM To: toasters(a)mathworks.com Subject: Re: Backups of Snapshots Kevin, This isn't what Jack's trying to acheive. However, besides that it's a dangerous recommendation. Automatic scheduled snapshots are rotated out by their naming convention. i.e. "nightly.0" is always last night's midnight snapshot. So, if for some reason the backup takes more than 24 hours to write to tape, or is delayed by a sheduling issue or a downed drive or whatever, you run the risk of a temporally inconsistent backup on tape after the snapshot rotation occurs. This might be made obvious by errors occuring because a file that was in the original directory map and was deleted during the backup disappear from the "nightly.0" snapshot when it's rotated for the next night (the previous night becomes "nightly.1"). i.e. you should always backup using the automatic "snapshot for backup.x" created by the NDMP client in Data OnTap, or use a manual snapshot. It is never wise to use the built-in scheduled snapshots for anything other than point-in-time restores from disk, because their contents will change at the next shedule period. I agree with your comment about using a static name. But you need both the name _and_ the *contents* to remain static, which isn't the case with the OnTap sheduled snapshots. A manual snapshot is always best for this, i.e. a script like: (using the snapshot name "dailybackup" rsh filer snap delete dailybackup ;(delete previous manual snapshot you've hung on to for restores) rsh filer snap create dailybackup backup /filer/.snapshot/dailybackup/ ;(or for Windows create a share that always points to the root of the snapshot and use that) regards, Alan McLachlan ___________________________________ Solution Architect - Data Centre Solutions Dimension Data Australia Alan.McLachlan(a)didata.com.au Tel. +61 (0)2 61225123 Fax +61 (0)2 62486346 Mobile 0428 655644 ----- Forwarded by Alan McLachlan on 05/10/2004 02:20 PM ----- "Kevin Mascarenhas" <KMascarenhas(a)syncsort.com> Sent by: owner-toasters(a)mathworks.com 01/10/2004 11:28 PM Please respond to "Kevin Mascarenhas" To: "Jack Lyons" <jack.lyons(a)martinagency.com> cc: <toasters(a)mathworks.com> Subject: Re: Backups of Snapshots Hi Jack, One other option you can consider is specifying an existing snapshot as your NDMP backup source. For example, if you configured your filer to take nightly snapshots at say 11:00pm and had your scheduled NDMP backups start at sometime after that, then you could configure the backup job to use the "nightly.0" snapshot as the backup source. When a snapshot is specified in this way, NDMP driven dump will not create another one for backup. Any backup software should allow you to define a job that uses an existing snapshot (or a snapshot that will exist at the time the job will run) for backup. Of course it's simplest if you configure the job to use a static snapshot name rather than one which will change each time the job is run so NetApp scheduled automatic snapshots will work perfectly. HTH. Kevin Kevin Mascarenhas KMascarenhas(a)syncsort.com Syncsort Inc. 50 Tice Blvd. Woodcliff Lake, NJ 07677 Tel: (201) 930-9700 http://www.syncsort.com ----- Original Message ----- From: "Jack Lyons" <jack.lyons(a)martinagency.com> To: "Paul Galjan" <galjan(a)gmail.com> Cc: <toasters(a)mathworks.com> Sent: Friday, October 01, 2004 6:45 AM Subject: Re: Backups of Snapshots > I guess that makes sense. it would have been nice to be able to look > restore a snapshot directory from 4 months ago! > > It looks like I could do a monthly snapshot manually and see how much space > it takes up. Right now, my snapshots are only taking up 23% of the space > allocated for snapshots. I am going to experiment with taking 6 "monthly > snapshots" > > Jack > ----- Original Message ----- > From: "Paul Galjan" <galjan(a)gmail.com> > To: "Jack Lyons" <jack.lyons(a)martinagency.com> > Cc: <toasters(a)mathworks.com> > Sent: Thursday, September 30, 2004 7:23 PM > Subject: Re: Backups of Snapshots > > > > Hi Jack, > > > > The snapshot directory is not included as part of the NDMP dump. When > > the filer is told to do an NDMP dump, it actually takes a snapshot, > > then dumps the data from that snapshot. > > > > --paul > > > > On Thu, 30 Sep 2004 18:41:11 -0400, Jack Lyons > > <jack.lyons(a)martinagency.com> wrote: > >> I had a user request a file from 4 months ago. The tape backups happened > >> at > >> 11:00 PM but the file they needed was earlier in the day. I was thinking > >> that the backups might have included the snapshot directory but it > >> didn't? > >> Is that configurable? > >> > >> Jack > >> > >> > > > ************************************************************************ ****** - NOTICE FROM DIMENSION DATA AUSTRALIA This message is confidential, and may contain proprietary or legally privileged information. If you have received this email in error, please notify the sender and delete it immediately. Internet communications are not secure. You should scan this message and any attachments for viruses. Under no circumstances do we accept liability for any loss or damage which may result from your receipt of this message or any attachments. ************************************************************************ ****** ***************************************************************************************************** If you have received this email in error please notify the the system manager or the sender immediately and do not disclosed the to anyone or make copies. ** eSafe scanned this email for viruses, vandals or malicious content ** ** To find out eSafe(TM) and other Aladdin Security Products, contact AMC Corporation at (212) 620-0700 *****************************************************************************************************

1 0

filer MRTG mods to see tape and disk IO, and consistent network io, and larger charts
by Tim McDonough 05 Oct '04

05 Oct '04

Hi Folks: Looking for general MRTG docs on how to modify the stock config to see: tape io for the filer ( read and write ) disk io .... larger charts consistent network info ( have gaps, and 'stuck points' where the network rate is the same for 3 hours ( sysstat confirms it's now ) Thanks

1 0

Re: Backups of Snapshots
by Alan.McLachlan＠didata.com.au 05 Oct '04

05 Oct '04

Kevin, This isn't what Jack's trying to acheive. However, besides that it's a dangerous recommendation. Automatic scheduled snapshots are rotated out by their naming convention. i.e. "nightly.0" is always last night's midnight snapshot. So, if for some reason the backup takes more than 24 hours to write to tape, or is delayed by a sheduling issue or a downed drive or whatever, you run the risk of a temporally inconsistent backup on tape after the snapshot rotation occurs. This might be made obvious by errors occuring because a file that was in the original directory map and was deleted during the backup disappear from the "nightly.0" snapshot when it's rotated for the next night (the previous night becomes "nightly.1"). i.e. you should always backup using the automatic "snapshot for backup.x" created by the NDMP client in Data OnTap, or use a manual snapshot. It is never wise to use the built-in scheduled snapshots for anything other than point-in-time restores from disk, because their contents will change at the next shedule period. I agree with your comment about using a static name. But you need both the name _and_ the *contents* to remain static, which isn't the case with the OnTap sheduled snapshots. A manual snapshot is always best for this, i.e. a script like: (using the snapshot name "dailybackup" rsh filer snap delete dailybackup ;(delete previous manual snapshot you've hung on to for restores) rsh filer snap create dailybackup backup /filer/.snapshot/dailybackup/ ;(or for Windows create a share that always points to the root of the snapshot and use that) regards, Alan McLachlan ___________________________________ Solution Architect - Data Centre Solutions Dimension Data Australia Alan.McLachlan(a)didata.com.au Tel. +61 (0)2 61225123 Fax +61 (0)2 62486346 Mobile 0428 655644 ----- Forwarded by Alan McLachlan on 05/10/2004 02:20 PM ----- "Kevin Mascarenhas" <KMascarenhas(a)syncsort.com> Sent by: owner-toasters(a)mathworks.com 01/10/2004 11:28 PM Please respond to "Kevin Mascarenhas" To: "Jack Lyons" <jack.lyons(a)martinagency.com> cc: <toasters(a)mathworks.com> Subject: Re: Backups of Snapshots Hi Jack, One other option you can consider is specifying an existing snapshot as your NDMP backup source. For example, if you configured your filer to take nightly snapshots at say 11:00pm and had your scheduled NDMP backups start at sometime after that, then you could configure the backup job to use the "nightly.0" snapshot as the backup source. When a snapshot is specified in this way, NDMP driven dump will not create another one for backup. Any backup software should allow you to define a job that uses an existing snapshot (or a snapshot that will exist at the time the job will run) for backup. Of course it's simplest if you configure the job to use a static snapshot name rather than one which will change each time the job is run so NetApp scheduled automatic snapshots will work perfectly. HTH. Kevin Kevin Mascarenhas KMascarenhas(a)syncsort.com Syncsort Inc. 50 Tice Blvd. Woodcliff Lake, NJ 07677 Tel: (201) 930-9700 http://www.syncsort.com ----- Original Message ----- From: "Jack Lyons" <jack.lyons(a)martinagency.com> To: "Paul Galjan" <galjan(a)gmail.com> Cc: <toasters(a)mathworks.com> Sent: Friday, October 01, 2004 6:45 AM Subject: Re: Backups of Snapshots > I guess that makes sense. it would have been nice to be able to look > restore a snapshot directory from 4 months ago! > > It looks like I could do a monthly snapshot manually and see how much space > it takes up. Right now, my snapshots are only taking up 23% of the space > allocated for snapshots. I am going to experiment with taking 6 "monthly > snapshots" > > Jack > ----- Original Message ----- > From: "Paul Galjan" <galjan(a)gmail.com> > To: "Jack Lyons" <jack.lyons(a)martinagency.com> > Cc: <toasters(a)mathworks.com> > Sent: Thursday, September 30, 2004 7:23 PM > Subject: Re: Backups of Snapshots > > > > Hi Jack, > > > > The snapshot directory is not included as part of the NDMP dump. When > > the filer is told to do an NDMP dump, it actually takes a snapshot, > > then dumps the data from that snapshot. > > > > --paul > > > > On Thu, 30 Sep 2004 18:41:11 -0400, Jack Lyons > > <jack.lyons(a)martinagency.com> wrote: > >> I had a user request a file from 4 months ago. The tape backups happened > >> at > >> 11:00 PM but the file they needed was earlier in the day. I was thinking > >> that the backups might have included the snapshot directory but it > >> didn't? > >> Is that configurable? > >> > >> Jack > >> > >> > > > ****************************************************************************** - NOTICE FROM DIMENSION DATA AUSTRALIA This message is confidential, and may contain proprietary or legally privileged information. If you have received this email in error, please notify the sender and delete it immediately. Internet communications are not secure. You should scan this message and any attachments for viruses. Under no circumstances do we accept liability for any loss or damage which may result from your receipt of this message or any attachments. ******************************************************************************

1 0

Undeliverable: Re:
by System Administrator 04 Oct '04

04 Oct '04

Your message To: Malek.hamdan Subject: Re: Sent: Tue, 5 Oct 2004 05:27:00 +1000 did not reach the following recipient(s): Malek.hamdan on Tue, 5 Oct 2004 04:27:08 +1000 The recipient name is not recognized The MTS-ID of the original message is: c=au;a=telememo ;p=davnet pty. ltd.;l=SYD-MAIL-0104100418274AVY7L3N MSEXCH:IMS:Davnet Pty. Ltd.:Sydney:SYD-MAIL-01 0 (000C05A6) Unknown Recipient

1 0

Undeliverable: Re:
by System Administrator 02 Oct '04

02 Oct '04

Your message To: Malek.hamdan Subject: Re: Sent: Sat, 2 Oct 2004 11:03:06 +1000 did not reach the following recipient(s): Malek.hamdan on Sat, 2 Oct 2004 10:03:17 +1000 The recipient name is not recognized The MTS-ID of the original message is: c=au;a=telememo ;p=davnet pty. ltd.;l=SYD-MAIL-0104100200034AVY691J MSEXCH:IMS:Davnet Pty. Ltd.:Sydney:SYD-MAIL-01 0 (000C05A6) Unknown Recipient

1 0

Undeliverable: Re:
by System Administrator 01 Oct '04

01 Oct '04

Your message To: Malek.hamdan Subject: Re: Sent: Sat, 2 Oct 2004 06:51:59 +1000 did not reach the following recipient(s): Malek.hamdan on Sat, 2 Oct 2004 05:52:08 +1000 The recipient name is not recognized The MTS-ID of the original message is: c=au;a=telememo ;p=davnet pty. ltd.;l=SYD-MAIL-0104100119524AVY682M MSEXCH:IMS:Davnet Pty. Ltd.:Sydney:SYD-MAIL-01 0 (000C05A6) Unknown Recipient

1 0

RE: Backups of Snapshots
by Jack Lyons 01 Oct '04

01 Oct '04

Thanks for the idea. My goal was to capture the snapshot subdirectory they way it was on that day. Specifying the snapshot I want to backup would be no different than selecting a different time to run the backups. Jack -----Original Message----- From: Kevin Mascarenhas [mailto:KMascarenhas@syncsort.com] Sent: Friday, October 01, 2004 9:29 AM To: Jack Lyons Cc: toasters(a)mathworks.com Subject: Re: Backups of Snapshots Hi Jack, One other option you can consider is specifying an existing snapshot as your NDMP backup source. For example, if you configured your filer to take nightly snapshots at say 11:00pm and had your scheduled NDMP backups start at sometime after that, then you could configure the backup job to use the "nightly.0" snapshot as the backup source. When a snapshot is specified in this way, NDMP driven dump will not create another one for backup. Any backup software should allow you to define a job that uses an existing snapshot (or a snapshot that will exist at the time the job will run) for backup. Of course it's simplest if you configure the job to use a static snapshot name rather than one which will change each time the job is run so NetApp scheduled automatic snapshots will work perfectly. HTH. Kevin Kevin Mascarenhas KMascarenhas(a)syncsort.com Syncsort Inc. 50 Tice Blvd. Woodcliff Lake, NJ 07677 Tel: (201) 930-9700 http://www.syncsort.com ----- Original Message ----- From: "Jack Lyons" <jack.lyons(a)martinagency.com> To: "Paul Galjan" <galjan(a)gmail.com> Cc: <toasters(a)mathworks.com> Sent: Friday, October 01, 2004 6:45 AM Subject: Re: Backups of Snapshots > I guess that makes sense. it would have been nice to be able to look > restore a snapshot directory from 4 months ago! > > It looks like I could do a monthly snapshot manually and see how much space > it takes up. Right now, my snapshots are only taking up 23% of the space > allocated for snapshots. I am going to experiment with taking 6 "monthly > snapshots" > > Jack > ----- Original Message ----- > From: "Paul Galjan" <galjan(a)gmail.com> > To: "Jack Lyons" <jack.lyons(a)martinagency.com> > Cc: <toasters(a)mathworks.com> > Sent: Thursday, September 30, 2004 7:23 PM > Subject: Re: Backups of Snapshots > > > > Hi Jack, > > > > The snapshot directory is not included as part of the NDMP dump. When > > the filer is told to do an NDMP dump, it actually takes a snapshot, > > then dumps the data from that snapshot. > > > > --paul > > > > On Thu, 30 Sep 2004 18:41:11 -0400, Jack Lyons > > <jack.lyons(a)martinagency.com> wrote: > >> I had a user request a file from 4 months ago. The tape backups happened > >> at > >> 11:00 PM but the file they needed was earlier in the day. I was thinking > >> that the backups might have included the snapshot directory but it > >> didn't? > >> Is that configurable? > >> > >> Jack > >> > >> > > > This email and its contents may be confidential. If it is and you are not the intended recipient, please do not disclose or use the information within this email or its attachments. If you have received this email in error, please delete it immediately. Thank you.

1 0

RE: iscsi benchmark tools
by Ben Rockwood 01 Oct '04

01 Oct '04

iSCSI benchmarking is difficult due to all the various factors that play into it, just like NAS. I'm a UNIX only user, but I'd recommend the 3 best tools out there: 1) IOzone: http://iozone.org/ 2) Bonnie++: http://www.coker.com.au/bonnie++/ 3) PostMark: http://www.netapp.com/tech_library/3022.html Bonnie++ is the best of the real quick and easy benchmarks around. IOZone is the best way to see whats really happening and where the sweet spot is, but more time consuming and confusing for the uninitiated. And PostMark is NetApp's own benchmark which does some interesting tests better representing the real world. If your using a software initiator make sure to carefully examine CPU and interupt activity on your system during the benchmarking session. benr. PS: Here is an example of Bonnie++. I ran this from my Linux desktop system (with X and 20 terms, etc running). I'm using JFS on an iSCSI LUN provided by an F840 with a single shelf of 36G drives. I'm using a Linux software initiator and using the public switched 100M ethernet that I use for everything else. This test used a 2G sample. Here's the numbers: [benr@nexus benchmark]$ bonnie++ -d /na_iscsi/bench_scratch/ -s 2048M -m "F840 iSCSI" (Lines removed) Version 1.03 ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP F840 iSCSI 2G 3802 57 11296 12 5330 5 4322 69 10489 5 290.0 2 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 3842 32 +++++ +++ 3120 21 2000 35 +++++ +++ 1986 24 -----Original Message----- From: owner-toasters(a)mathworks.com on behalf of Bui, Marcus Sent: Tue 9/28/2004 1:08 PM To: toasters(a)mathworks.com Cc: Subject: iscsi benchmark tools Hi Toasters, I just setup a iSCSI test environment and comparing the performance with a server's local disk. The target is the R200 (50GB LUN) with a W2K initiator, MS's iSCSI driver on 100MB NIC (Interl Ethernet NIC not iSCSI HBA). I used two different freeware tools (Roadkil Speed Disk v1.1 and IOMeter v200.7.30) and get totally different results. iSCSI volume Random Read: Roadkil - 30.3MB/sec IOMeter - .06MB/sec iSCSI volume Write IOMeter - .03MB/sec Roadkil - N/A on freeware version Based on the Random Read numbers, it seems the two results are vastly different. Don't know which one depicts the 'true' performance for the iSCSI volume. Can someone recommend a tool you use before to benchmark iSCIS performance? Cheers, Marcus Bui AIM Investments IT Services phone: 713-986-9061 fax: 713-821-9524 email: marcus.bui(a)aiminvestments.com pager: 8773450839(a)archwireless.net ----------------------------------------- Confidentiality Note: The information contained in this message, and any attachments, may contain confidential and/or privileged material. It is intended solely for the person or entity to which it is addressed. Any review, retransmission, dissemination, or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer.

4 4

RE: Backups of Snapshots
by McCarthy, Tim 01 Oct '04

01 Oct '04

It probably would not be that much different than keep 4 weeklies on line for a month.... One month back, keep 4 weeklies. Two months back, keep 8 weeklies. Etc... Since they are all differential anyways, there should not be much delta between them...unless of course there was a lot of deleting in some of those weeks. --tmac ====================== Tim McCarthy Professional Services/Systems Engineer NetApp Federal Systems, Inc. 410-551-3970 (o) 443-363-0208 (f) tmac(a)netapp.com tmac-pager(a)netapp.com ====================== -----Original Message----- From: Jack Lyons [mailto:jack.lyons@martinagency.com] Sent: Friday, October 01, 2004 6:46 AM To: Paul Galjan Cc: toasters(a)mathworks.com Subject: Re: Backups of Snapshots I guess that makes sense. it would have been nice to be able to look restore a snapshot directory from 4 months ago! It looks like I could do a monthly snapshot manually and see how much space it takes up. Right now, my snapshots are only taking up 23% of the space allocated for snapshots. I am going to experiment with taking 6 "monthly snapshots" Jack ----- Original Message ----- From: "Paul Galjan" <galjan(a)gmail.com> To: "Jack Lyons" <jack.lyons(a)martinagency.com> Cc: <toasters(a)mathworks.com> Sent: Thursday, September 30, 2004 7:23 PM Subject: Re: Backups of Snapshots > Hi Jack, > > The snapshot directory is not included as part of the NDMP dump. When > the filer is told to do an NDMP dump, it actually takes a snapshot, > then dumps the data from that snapshot. > > --paul > > On Thu, 30 Sep 2004 18:41:11 -0400, Jack Lyons > <jack.lyons(a)martinagency.com> wrote: >> I had a user request a file from 4 months ago. The tape backups happened >> at >> 11:00 PM but the file they needed was earlier in the day. I was thinking >> that the backups might have included the snapshot directory but it >> didn't? >> Is that configurable? >> >> Jack >> >> >

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

toasters October 2004