Donald,
You describe this as a specific problem - nevertheless, are we sure this is not just working as designed? All these FlexVols are located in the same aggregate - so parallel dumps from the same aggregate going slower makes sense to me. dump is pretty efficient once it reaches the later phases - I would suggest you to debug this in a different method - use the PerfStat tool to capture the statit and sysstat outputs, and review whether the disks are highly utilized during a single dump. If this is the case already, then there's nothing wrong - you're already maxing out your shelves I/O by this dump. Unless the FAS960 platform is maxed by itself (which I guestimate to NOT be the case), adding shelves would actually help you achieve faster dumps, not slower.
My side non-proven suggestion - stage your NDMP backups so that the first phases will not all occur in parallel. This may help improve your total backup time.
Anyhow, I'd love to see numbers when I hear a problem such as yours - what's your ONTAP? how many MB/sec are you seeing in sysstat etc...
Eyal. http://filers.blogspot.com http://stupidstorage.blogspot.com
On 10/10/06, Glascock, Donald glascock.donald@mayo.edu wrote:
Hi, Toaster folks --
We're a small research group on our third Filer, which is a single FAS960 with five shelves on two cross-connected 2-Gb loops. We trigger flexVolume-level dumps of our one aggregate directly to 2-Gb-fibre-connected LTO-3 drives.
When we dump two volumes simultaneously to two tape drives, the time taken to dump each volume increases by a third to a half again, compared to dumping each volume independently. For example, a stand-alone dump may take two hours, but if another dump of another volume to another drive is run simultaneously, then that two-hour dump becomes a three-hour dump. Adding a third simultaneous dump to a third drive causes an even greater increase in dump times.
For diagnostic purposes, I've launched a conventional dump of one volume triggered via NDMP our backup software, waited for the dump to reach Phase-IV (the writing-data phase), and launched from the FAS960's console a dump of another volume to the FAS960's null device. Soon after this second dump starts, the write speed on the tape drive associated with the first dump falls off by quite a bit. This write speed picks up instantly when I kill the second dump. Since the second dump was only in Phase-I (inspecting files, but not writing them), and since this second dump was to the local null device anyway, I'm led to think that the performance issues lie somewhere within the Filer's head and Disk I/O subsystem.
With one-at-a-time backups of our volumes, our full backup window is about eight hours (for about two terabytes). We were planning to buy two more shelves in early 2007.
Should I expect a FAS960 to handle two or three simultaneous full dumps without a significant loss in dump speed? Are you getting nearly-linear performance out of your simultaneous dumps?
Thanks for your time & have a great day!
Don Glascock
-- Donald S. Glascock Mailstop RO-SN-2-SPPDG Special Purpose Processor Development Group Mayo Foundation 4001 41st Street NW Rochester, MN 55901 Glascock.Donald@Mayo.EDU +1.507.538.5467 +1.507.284.9171 (fax) http://www.mayo.edu/sppdg/ "No matter where you go, there you are."