Are you doing Qtree backups or volume backups? Volume backups perform significantly faster than Qtree backups.
-----Original Message----- From: owner-toasters@mathworks.com [mailto:george@YorkU.CA] Sent: Tuesday, April 04, 2006 2:11 PM To: gdekhayser@voyantinc.com Cc: toasters@mathworks.com Subject: Re: volume space & ndmp backups issues
Glenn, I agree snapmirror would be the preferable way of doing this. However, this is not an option at this time.
The backups are going to an ADIC I2000 tape library with IBM ULTRIUM LTO3 tape drives. Each filer has its own dedicated tape drive in this library connected via a fiber.
George
On Tue, 4 Apr 2006 12:05:43 -0400, "Glenn Dekhayser" gdekhayser@voyantinc.com wrote:
I would think that a good portion of your backup time is the ndmp scan prior to the backup. Not sure anything will help with that.
I would think the best way to accomplish backup in the shortest time is to do volume level snapmirrors, which don't care about files. But, barring that-
Is your NDMP backup going to a FC tape library? Or SCSI-attached? Or Three way?
Also, what onTAP are you running? Perhaps if you went with a large aggregate you'd be better off in all cases.
Glenn (the other one)
-----Original Message----- From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of George Kahler Sent: Tuesday, April 04, 2006 10:45 AM To: toasters@mathworks.com Subject: volume space & ndmp backups issues
I have a space and a backup (ndmp) issues with one of my volumes that is dedicated for mail in 'maildir' format (one message per file).
I'm looking for suggestions from people on alleviating the space issue and possibly suggestions on the ndmp backup of this volume. I'm thinking that these two might be related.
filer#1:/vol/maildir (14 x 72 GB disks) 2 qtrees: ugrad (undergrad students) grad (grad students)
backup: (nightly incr. on an average backs up about) 11GB , 6million files, runs for 7 hrsfiler#2:/vol/maildir2 (14 x 72 GB disks) 1 qtree: emp (employees) backup: (nightly incr. on an average backs up about) 8GB , 5million files, runs for 1.5 hrs
All 3 qtrees have the same 2 level directory tree structure (as suggested by NetApp) and a common hashing algorithm is used to create 'home directories' in these qtrees. The top directory contains sub-directories from 0-to-f. Each of those directories contain sub-directories ranging from 00-to-ff. So a persons home directory would be something like: /vol/maildir/grad/2/75/dummy
I need more space for the filer#1:/vol/maildir volume.
Q1: should I just expand the volume onto more disks ? having more spindles might speed up the backup ?
Q2: should I separate the 2 qtrees (u/grad) into their own volumes (14 x 72 GB each) to give each more space ? having only one qtree to scan , ndmp might run that much faster ?
Thanks for any suggestions. George
Entire volume at a time: /vol/maildir
George
On Tue, 4 Apr 2006 13:13:15 -0400 , "Holland, William L" hollandwl@state.gov wrote:
Are you doing Qtree backups or volume backups? Volume backups perform significantly faster than Qtree backups.
-----Original Message----- From: owner-toasters@mathworks.com [mailto:george@YorkU.CA] Sent: Tuesday, April 04, 2006 2:11 PM To: gdekhayser@voyantinc.com Cc: toasters@mathworks.com Subject: Re: volume space & ndmp backups issues
Glenn, I agree snapmirror would be the preferable way of doing this. However, this is not an option at this time.
The backups are going to an ADIC I2000 tape library with IBM ULTRIUM LTO3 tape drives. Each filer has its own dedicated tape drive in this library connected via a fiber.
George
On Tue, 4 Apr 2006 12:05:43 -0400, "Glenn Dekhayser" gdekhayser@voyantinc.com wrote:
I would think that a good portion of your backup time is the ndmp scan prior to the backup. Not sure anything will help with that.
I would think the best way to accomplish backup in the shortest time is to do volume level snapmirrors, which don't care about files. But, barring that-
Is your NDMP backup going to a FC tape library? Or SCSI-attached? Or Three way?
Also, what onTAP are you running? Perhaps if you went with a large aggregate you'd be better off in all cases.
Glenn (the other one)
-----Original Message----- From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of George Kahler Sent: Tuesday, April 04, 2006 10:45 AM To: toasters@mathworks.com Subject: volume space & ndmp backups issues
I have a space and a backup (ndmp) issues with one of my volumes that is dedicated for mail in 'maildir' format (one message per file).
I'm looking for suggestions from people on alleviating the space issue and possibly suggestions on the ndmp backup of this volume. I'm thinking that these two might be related.
filer#1:/vol/maildir (14 x 72 GB disks) 2 qtrees: ugrad (undergrad students) grad (grad students)
backup: (nightly incr. on an average backs up about) 11GB , 6million files, runs for 7 hrsfiler#2:/vol/maildir2 (14 x 72 GB disks) 1 qtree: emp (employees) backup: (nightly incr. on an average backs up about) 8GB , 5million files, runs for 1.5 hrs
All 3 qtrees have the same 2 level directory tree structure (as suggested by NetApp) and a common hashing algorithm is used to create 'home directories' in these qtrees. The top directory contains sub-directories from 0-to-f. Each of those directories contain sub-directories ranging from 00-to-ff. So a persons home directory would be something like: /vol/maildir/grad/2/75/dummy
I need more space for the filer#1:/vol/maildir volume.
Q1: should I just expand the volume onto more disks ? having more spindles might speed up the backup ?
Q2: should I separate the 2 qtrees (u/grad) into their own volumes (14 x 72 GB each) to give each more space ? having only one qtree to scan , ndmp might run that much faster ?
Thanks for any suggestions. George
-
George Kahler -
Holland, William L