Here's a good document to browse over for this particular issue...
http://www.netapp.com/library/tr/3437.pdf
________________________________
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Jason Herring Sent: Monday, March 19, 2007 3:08 PM To: Sphar, Mike; toasters@mathworks.com Subject: RE: FAS3020 - aggr best practises.
You can always go with a larger RAID group size - it does support up to 28. However, at some point you have to bite the bullet and lose 2 more disks to parity. It depends on how you want the math to work out in the long run.
The reason you want to add the disks in large sets is so you have a more level writing of the data to the disks - you don't want to get 3-4 'hot disks' slowing the whole aggregate down until the data is spread evenly among the aggregate.'s new disks...
-----Original Message----- From: owner-toasters@mathworks.com on behalf of Sphar, Mike Sent: Mon 3/19/2007 11:06 AM To: toasters@mathworks.com Cc: Subject: RE: FAS3020 - aggr best practises.
Point one reminds me of a question I've been pondering. I've got a filer using RAID-DP with 3X14-disk raid groups, and currently 7 spares. Ideally I'd only keep two spares, but I'm still not clear on the pros/cons of adding a 5 disk raid group, effectively only adding three more data disks to the volume.
I'm not in a space crunch currently but I certainly will be at some point. Is it best to leave so many extra spares until I can add a full raid group all at once? Or in my case is it not that important?
--
Michael W. Sphar - IS&T - Lead Systems Administrator
SMBU Engineering Support Services, BMC Software
________________________________
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Learmonth, Peter Sent: Saturday, March 17, 2007 6:43 PM To: Darish Rajanayagam; toasters@mathworks.com Subject: RE: FAS3020 - aggr best practises.
Hi Darish
Welcome to NetApp and to Toasters!
1. You can do an "aggr add aggr0 56" or use the FilerView GUI and add all 56 of the new disks into the existing aggregate. You can physically add the shelves and add them to the aggr while the filer is up and running. I see no disadvantages, and that is the best practice. (Add disks in large sets, ideally the raid group size).
I've read this and other similar documents, but it doesn't really answer my specific question. Basically:
Company purchases filer with 4 full shelves of 14 disks.
Netapp recommends 16-disk raid groups at install time.
Aggregate is created using 3 full raid groups of 16 disks, per best practices recommendations of adding full raid groups
Filer is left with 8 spares (not 7 as I incorrectly mentioned before)
My options at this point seem to be: 1) add another raid group of 6 disks or 2) leave 8 spares until some future time when capital budgets may purchase more shelves. I don't think increasing the raid group size is an option as I don't believe you can increase the size of existing already-full raid groups?
Anyway, it feels like a big waste to me to have 8 hot spares, but if the performance or reliability costs of a 6 disk RAID-DP group are too large I can live with it.
Seems to me your best bet would be to have 18 disk raid groups (or maybe 17, so you have a happy medium, personally if I can get the money I like a HSP per shelf). ________________________________
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Sphar, Mike Sent: Monday, March 19, 2007 5:02 PM To: toasters@mathworks.com Subject: RE: FAS3020 - aggr best practises.
I've read this and other similar documents, but it doesn't really answer my specific question. Basically:
Company purchases filer with 4 full shelves of 14 disks.
Netapp recommends 16-disk raid groups at install time.
Aggregate is created using 3 full raid groups of 16 disks, per best practices recommendations of adding full raid groups
Filer is left with 8 spares (not 7 as I incorrectly mentioned before)
My options at this point seem to be: 1) add another raid group of 6 disks or 2) leave 8 spares until some future time when capital budgets may purchase more shelves. I don't think increasing the raid group size is an option as I don't believe you can increase the size of existing already-full raid groups?
Anyway, it feels like a big waste to me to have 8 hot spares, but if the performance or reliability costs of a 6 disk RAID-DP group are too large I can live with it.
You can always go with smaller raid groups too.
4 shelves * 14 disks = 56 disks 56 - 2 hot spares = 54 disks
54 / 3 = 18 disks per group 54 / 4 = 14 + 14 + 13 + 13 54 / 5 = 11 + 11 + 11 + 11 + 10
You can use any of the above:
aggr create aggr1 -t raid_dp -r 14 28 aggr options aggr1 13 aggr add aggr1 26
or
aggr create aggr1 -t raid_dp -r 11 44 aggr options aggr1 10 aggr add aggr1 10
Any of these methods allows you to go back and add disks to any of the raid groups at a later date: aggr options aggr1 raidsize 16 aggr add -f -g rg0 xx (where xx is the difference of 16 and the current raid group)
I do it all the time. This way I always have even raid groups.
On 3/20/07, Page, Jeremy jeremy.page@gilbarco.com wrote:
Seems to me your best bet would be to have 18 disk raid groups (or maybe 17, so you have a happy medium, personally if I can get the money I like a HSP per shelf). ________________________________ From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Sphar, Mike Sent: Monday, March 19, 2007 5:02 PM To: toasters@mathworks.com
Subject: RE: FAS3020 - aggr best practises.
I've read this and other similar documents, but it doesn't really answer my specific question. Basically:
Company purchases filer with 4 full shelves of 14 disks.
Netapp recommends 16-disk raid groups at install time.
Aggregate is created using 3 full raid groups of 16 disks, per best practices recommendations of adding full raid groups
Filer is left with 8 spares (not 7 as I incorrectly mentioned before)
My options at this point seem to be: 1) add another raid group of 6 disks or 2) leave 8 spares until some future time when capital budgets may purchase more shelves. I don't think increasing the raid group size is an option as I don't believe you can increase the size of existing already-full raid groups?
Anyway, it feels like a big waste to me to have 8 hot spares, but if the performance or reliability costs of a 6 disk RAID-DP group are too large I can live with it.
--
Michael W. Sphar - IS&T - Lead Systems Administrator
SMBU Engineering Support Services, BMC Software
From: Parisi, Justin [mailto:Justin.Parisi@netapp.com] Sent: Monday, March 19, 2007 3:01 PM To: Jason Herring; Sphar, Mike; toasters@mathworks.com Subject: RE: FAS3020 - aggr best practises.
Here's a good document to browse over for this particular issue...
http://www.netapp.com/library/tr/3437.pdf
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Jason Herring Sent: Monday, March 19, 2007 3:08 PM To: Sphar, Mike; toasters@mathworks.com Subject: RE: FAS3020 - aggr best practises.
You can always go with a larger RAID group size - it does support up to 28. However, at some point you have to bite the bullet and lose 2 more disks to parity. It depends on how you want the math to work out in the long run.
The reason you want to add the disks in large sets is so you have a more level writing of the data to the disks - you don't want to get 3-4 'hot disks' slowing the whole aggregate down until the data is spread evenly among the aggregate.'s new disks...
-----Original Message----- From: owner-toasters@mathworks.com on behalf of Sphar, Mike Sent: Mon 3/19/2007 11:06 AM To: toasters@mathworks.com Cc: Subject: RE: FAS3020 - aggr best practises.
Point one reminds me of a question I've been pondering. I've got a filer using RAID-DP with 3X14-disk raid groups, and currently 7 spares. Ideally I'd only keep two spares, but I'm still not clear on the pros/cons of adding a 5 disk raid group, effectively only adding three more data disks to the volume.
I'm not in a space crunch currently but I certainly will be at some point. Is it best to leave so many extra spares until I can add a full raid group all at once? Or in my case is it not that important?
--
Michael W. Sphar - IS&T - Lead Systems Administrator
SMBU Engineering Support Services, BMC Software
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Learmonth, Peter Sent: Saturday, March 17, 2007 6:43 PM To: Darish Rajanayagam; toasters@mathworks.com Subject: RE: FAS3020 - aggr best practises.
Hi Darish
Welcome to NetApp and to Toasters!
- You can do an "aggr add aggr0 56" or use the FilerView GUI and add
all 56 of the new disks into the existing aggregate. You can physically add the shelves and add them to the aggr while the filer is up and running. I see no disadvantages, and that is the best practice. (Add disks in large sets, ideally the raid group size).
This message (including any attachments) contains confidential and/or proprietary information intended only for the addressee. Any unauthorized disclosure, copying, distribution or reliance on the contents of this information is strictly prohibited and may constitute a violation of law. If you are not the intended recipient, please notify the sender immediately by responding to this e-mail, and delete the message from your system. If you have any questions about this e-mail please notify the sender immediately.
Mike_Sphar@bmc.com writes:
I've read this and other similar documents, but it doesn't really answer my specific question. Basically:
Company purchases filer with 4 full shelves of 14 disks.
Netapp recommends 16-disk raid groups at install time.
Aggregate is created using 3 full raid groups of 16 disks, per best practices recommendations of adding full raid groups
Filer is left with 8 spares (not 7 as I incorrectly mentioned before)
My options at this point seem to be: 1) add another raid group of 6 disks or 2) leave 8 spares until some future time when capital budgets may purchase more shelves. I don't think increasing the raid group size is an option as I don't believe you can increase the size of existing already-full raid groups?
If you are using RAID_DP then 16-disk raid groups won't be the largest that ONTAP allows, will they? You can do
aggr options [name] raidsize 18
and then add two disks to each existing raid group with
aggr add [name] -f -g rg0 2 aggr add [name] -f -g rg1 2 aggr add [name] -f -g rg2 2
(always supposing you don't need to retain the ability to revert to a pre-6.2 ONTAP: see the notes about the -g option in the na_aggr man page)
Whether that's desirable or not, I'm not sure, but it should at least be possible.
I've read this and other similar documents, but it doesn't really answer my specific question. Basically:
Company purchases filer with 4 full shelves of 14 disks.
Netapp recommends 16-disk raid groups at install time.
Aggregate is created using 3 full raid groups of 16 disks, per best practices recommendations of adding full raid groups
Filer is left with 8 spares (not 7 as I incorrectly mentioned before)
If you're disks are 72 GB or 144GB, then expanding the RAID groups to 20-26 disks shouldn't be that big of a deal.
Large RAID groups are primarily a concern when discussing rebuild times in case of disk failure.. with small disks, the risk is fairly minimal.
Regards, Max
-
Chris Thompson -
Max -
Page, Jeremy -
Parisi, Justin -
Sphar, Mike -
tmac