Hello toasters,
A few weeks ago, I had the opportunity to destroy an aggregate in order to consolidate capacity. I started off with two aggregates, each with two RG and both aggr were 600GB drives only the older drives were FC/15K whereas the new aggregate is SAS/10K (I know you can mix RPMs and I'm ok with that). What I'm not ok with is what happened when I attempted to add 8 drives to the new aggregate. The current RG size was 12 and there were 12 drives in the first RG and 11 in the second, I then changed the RG size to 16 and added the drives to the aggregate (expecting to add 4 in each RG). But what really happened is that OnTap added 5 drives to RG1 and created RG2 with the remaining 3 drives.
Support couldn't find any documentation on this and (I knew this) that it could only be corrected by destroying the aggregate. Have any of you experienced this?
Thanks, Bill
-Long time lurker, first time poster
===================================
Please consider the environment before printing this e-mail
Cleveland Clinic is ranked as one of the top hospitals in America by U.S.News & World Report (2014). Visit us online at http://www.clevelandclinic.org for a complete listing of our services, staff and locations.
Confidentiality Note: This message is intended for use only by the individual or entity to which it is addressed and may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient or the employee or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy.
Thank you.
I think by default it only adds disks to the 'last' (or highest number raid group) . You have to use the -g <raid-group> option to add to 'other' raid groups.
So if you changed the aggr size to 16, you then want to add 4 disks that rg0, then add 4 disks to rg1.
Then you want to run reallocate on the volumes.
--rdp
From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Woods, Bill Sent: Wednesday, February 25, 2015 2:15 PM To: toasters@teaparty.net Subject: Adding disks to existing aggregates
Hello toasters,
A few weeks ago, I had the opportunity to destroy an aggregate in order to consolidate capacity. I started off with two aggregates, each with two RG and both aggr were 600GB drives only the older drives were FC/15K whereas the new aggregate is SAS/10K (I know you can mix RPMs and I'm ok with that). What I'm not ok with is what happened when I attempted to add 8 drives to the new aggregate. The current RG size was 12 and there were 12 drives in the first RG and 11 in the second, I then changed the RG size to 16 and added the drives to the aggregate (expecting to add 4 in each RG). But what really happened is that OnTap added 5 drives to RG1 and created RG2 with the remaining 3 drives.
Support couldn't find any documentation on this and (I knew this) that it could only be corrected by destroying the aggregate. Have any of you experienced this?
Thanks, Bill
-Long time lurker, first time poster
=================================== Please consider the environment before printing this e-mail Cleveland Clinic is ranked as one of the top hospitals in America by U.S.News & World Report (2014). Visit us online at http://www.clevelandclinic.org for a complete listing of our services, staff and locations. Confidentiality Note: This message is intended for use only by the individual or entity to which it is addressed and may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient or the employee or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. Thank you.
Yep, also I know you said it was a new aggregate, but if it had data on it, you would want to run volume reallocates after or you would have hot spots on the new empty disks.
From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Payne, Richard Sent: Wednesday, February 25, 2015 2:25 PM To: Woods, Bill; toasters@teaparty.net Subject: RE: Adding disks to existing aggregates
I think by default it only adds disks to the 'last' (or highest number raid group) . You have to use the -g <raid-group> option to add to 'other' raid groups.
So if you changed the aggr size to 16, you then want to add 4 disks that rg0, then add 4 disks to rg1.
Then you want to run reallocate on the volumes.
--rdp
From: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Woods, Bill Sent: Wednesday, February 25, 2015 2:15 PM To: toasters@teaparty.netmailto:toasters@teaparty.net Subject: Adding disks to existing aggregates
Hello toasters,
A few weeks ago, I had the opportunity to destroy an aggregate in order to consolidate capacity. I started off with two aggregates, each with two RG and both aggr were 600GB drives only the older drives were FC/15K whereas the new aggregate is SAS/10K (I know you can mix RPMs and I'm ok with that). What I'm not ok with is what happened when I attempted to add 8 drives to the new aggregate. The current RG size was 12 and there were 12 drives in the first RG and 11 in the second, I then changed the RG size to 16 and added the drives to the aggregate (expecting to add 4 in each RG). But what really happened is that OnTap added 5 drives to RG1 and created RG2 with the remaining 3 drives.
Support couldn't find any documentation on this and (I knew this) that it could only be corrected by destroying the aggregate. Have any of you experienced this?
Thanks, Bill
-Long time lurker, first time poster
=================================== Please consider the environment before printing this e-mail Cleveland Clinic is ranked as one of the top hospitals in America by U.S.News & World Report (2014). Visit us online at http://www.clevelandclinic.org for a complete listing of our services, staff and locations. Confidentiality Note: This message is intended for use only by the individual or entity to which it is addressed and may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient or the employee or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. Thank you.
One of the rare cases where I really do not mind use the GUI (Filerview, System manager)
Go to the advanced areas (when adding disks) and it allows you to add disks to earlier raid groups.
--tmac
*Tim McCarthy*
On Wed, Feb 25, 2015 at 2:34 PM, Jordan Slingerland < Jordan.Slingerland@independenthealth.com> wrote:
Yep, also I know you said it was a new aggregate, but if it had data on it, you would want to run volume reallocates after or you would have hot spots on the new empty disks.
*From:* toasters-bounces@teaparty.net [mailto: toasters-bounces@teaparty.net] *On Behalf Of *Payne, Richard *Sent:* Wednesday, February 25, 2015 2:25 PM *To:* Woods, Bill; toasters@teaparty.net *Subject:* RE: Adding disks to existing aggregates
I think by default it only adds disks to the 'last' (or highest number raid group) . You have to use the -g <raid-group> option to add to 'other' raid groups.
So if you changed the aggr size to 16, you then want to add 4 disks that rg0, then add 4 disks to rg1.
Then you want to run reallocate on the volumes.
--rdp
*From:* toasters-bounces@teaparty.net [ mailto:toasters-bounces@teaparty.net toasters-bounces@teaparty.net] *On Behalf Of *Woods, Bill *Sent:* Wednesday, February 25, 2015 2:15 PM *To:* toasters@teaparty.net *Subject:* Adding disks to existing aggregates
Hello toasters,
A few weeks ago, I had the opportunity to destroy an aggregate in order to consolidate capacity. I started off with two aggregates, each with two RG and both aggr were 600GB drives only the older drives were FC/15K whereas the new aggregate is SAS/10K (I know you can mix RPMs and I'm ok with that). What I'm not ok with is what happened when I attempted to add 8 drives to the new aggregate. The current RG size was 12 and there were 12 drives in the first RG and 11 in the second, I then changed the RG size to 16 and added the drives to the aggregate (expecting to add 4 in each RG). But what really happened is that OnTap added 5 drives to RG1 and created RG2 with the remaining 3 drives.
Support couldn't find any documentation on this and (I knew this) that it could only be corrected by destroying the aggregate. Have any of you experienced this?
Thanks,
Bill
-Long time lurker, first time poster
===================================
Please consider the environment before printing this e-mail
Cleveland Clinic is ranked as one of the top hospitals in America by U.S.News & World Report (2014). Visit us online at http://www.clevelandclinic.org for a complete listing of our services, staff and locations. Confidentiality Note: This message is intended for use only by the individual or entity to which it is addressed and may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient or the employee or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. Thank you.
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Also, in newer version (8.1 up?) if you are adding disks to an existing aggregate, you usually get a prompt to view the proposed raid group layout and confirm it.
Did you not get that message? Are you on a not-so-new version of ontap?
--Jordan
From: tmac [mailto:tmacmd@gmail.com] Sent: Wednesday, February 25, 2015 2:43 PM To: Jordan Slingerland Cc: Payne, Richard; Woods, Bill; toasters@teaparty.net Subject: Re: Adding disks to existing aggregates
One of the rare cases where I really do not mind use the GUI (Filerview, System manager)
Go to the advanced areas (when adding disks) and it allows you to add disks to earlier raid groups.
--tmac
Tim McCarthy
On Wed, Feb 25, 2015 at 2:34 PM, Jordan Slingerland <Jordan.Slingerland@independenthealth.commailto:Jordan.Slingerland@independenthealth.com> wrote: Yep, also I know you said it was a new aggregate, but if it had data on it, you would want to run volume reallocates after or you would have hot spots on the new empty disks.
From: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net] On Behalf Of Payne, Richard Sent: Wednesday, February 25, 2015 2:25 PM To: Woods, Bill; toasters@teaparty.netmailto:toasters@teaparty.net Subject: RE: Adding disks to existing aggregates
I think by default it only adds disks to the 'last' (or highest number raid group) . You have to use the -g <raid-group> option to add to 'other' raid groups.
So if you changed the aggr size to 16, you then want to add 4 disks that rg0, then add 4 disks to rg1.
Then you want to run reallocate on the volumes.
--rdp
From: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Woods, Bill Sent: Wednesday, February 25, 2015 2:15 PM To: toasters@teaparty.netmailto:toasters@teaparty.net Subject: Adding disks to existing aggregates
Hello toasters,
A few weeks ago, I had the opportunity to destroy an aggregate in order to consolidate capacity. I started off with two aggregates, each with two RG and both aggr were 600GB drives only the older drives were FC/15K whereas the new aggregate is SAS/10K (I know you can mix RPMs and I'm ok with that). What I'm not ok with is what happened when I attempted to add 8 drives to the new aggregate. The current RG size was 12 and there were 12 drives in the first RG and 11 in the second, I then changed the RG size to 16 and added the drives to the aggregate (expecting to add 4 in each RG). But what really happened is that OnTap added 5 drives to RG1 and created RG2 with the remaining 3 drives.
Support couldn't find any documentation on this and (I knew this) that it could only be corrected by destroying the aggregate. Have any of you experienced this?
Thanks, Bill
-Long time lurker, first time poster
=================================== Please consider the environment before printing this e-mail Cleveland Clinic is ranked as one of the top hospitals in America by U.S.News & World Report (2014). Visit us online at http://www.clevelandclinic.org for a complete listing of our services, staff and locations. Confidentiality Note: This message is intended for use only by the individual or entity to which it is addressed and may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient or the employee or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. Thank you.
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Just to add to the slew of responses: Like Richard said, it's the default behaviour.
The man page says: If the -g option is not used, the disks are added _to the most recently created RAID group until it is full_, and _then one or more new RAID groups are created_ and the remaining disks are added to new groups. Any other _existing RAID groups_ that are not full _remain_ partially filled.
One of those cases where I actually prefer the GUI...
Sebastian
On 2/25/2015 8:24 PM, Payne, Richard wrote:
I think by default it only adds disks to the ‘last’ (or highest number raid group) . You have to use the –g <raid-group> option to add to ‘other’ raid groups.
So if you changed the aggr size to 16, you then want to add 4 disks that rg0, then add 4 disks to rg1.
Then you want to run reallocate on the volumes.
--rdp
*From:*toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] *On Behalf Of *Woods, Bill *Sent:* Wednesday, February 25, 2015 2:15 PM *To:* toasters@teaparty.net *Subject:* Adding disks to existing aggregates
Hello toasters,
A few weeks ago, I had the opportunity to destroy an aggregate in order to consolidate capacity. I started off with two aggregates, each with two RG and both aggr were 600GB drives only the older drives were FC/15K whereas the new aggregate is SAS/10K (I know you can mix RPMs and I’m ok with that). What I’m not ok with is what happened when I attempted to add 8 drives to the new aggregate. The current RG size was 12 and there were 12 drives in the first RG and 11 in the second, I then changed the RG size to 16 and added the drives to the aggregate (expecting to add 4 in each RG). But what really happened is that OnTap added 5 drives to RG1 and created RG2 with the remaining 3 drives.
Support couldn’t find any documentation on this and (I knew this) that it could only be corrected by destroying the aggregate. Have any of you experienced this?
Thanks,
Bill
-Long time lurker, first time poster
===================================
Please consider the environment before printing this e-mail
Cleveland Clinic is ranked as one of the top hospitals in America by U.S.News & World Report (2014). Visit us online at http://www.clevelandclinic.org for a complete listing of our services, staff and locations. Confidentiality Note: This message is intended for use only by the individual or entity to which it is addressed and may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient or the employee or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. Thank you.
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
To: Richard, I discovered that (too late) after the fact. Thanks.
To: Basil, RG0 had 12 drives, RG1 had 11 drives. I changed the RG size to 16 and added 8 drives, 5 of which added to RG1 and OnTap created RG2 with 3 drives. Thanks for the tip on previewing the RG layout. I'll keep that in mind.
To: Jordan, Thanks.
And to all of the others out there curious, there is currently sufficient capacity in the aggregate. Next year we intend to refresh (Clustered) that will eliminate the issue.
From: Payne, Richard [mailto:richard.payne@amd.com] Sent: Wednesday, February 25, 2015 2:25 PM To: Woods, Bill; toasters@teaparty.net Subject: RE: Adding disks to existing aggregates
I think by default it only adds disks to the 'last' (or highest number raid group) . You have to use the -g <raid-group> option to add to 'other' raid groups.
So if you changed the aggr size to 16, you then want to add 4 disks that rg0, then add 4 disks to rg1.
Then you want to run reallocate on the volumes.
--rdp
From: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Woods, Bill Sent: Wednesday, February 25, 2015 2:15 PM To: toasters@teaparty.netmailto:toasters@teaparty.net Subject: Adding disks to existing aggregates
Hello toasters,
A few weeks ago, I had the opportunity to destroy an aggregate in order to consolidate capacity. I started off with two aggregates, each with two RG and both aggr were 600GB drives only the older drives were FC/15K whereas the new aggregate is SAS/10K (I know you can mix RPMs and I'm ok with that). What I'm not ok with is what happened when I attempted to add 8 drives to the new aggregate. The current RG size was 12 and there were 12 drives in the first RG and 11 in the second, I then changed the RG size to 16 and added the drives to the aggregate (expecting to add 4 in each RG). But what really happened is that OnTap added 5 drives to RG1 and created RG2 with the remaining 3 drives.
Support couldn't find any documentation on this and (I knew this) that it could only be corrected by destroying the aggregate. Have any of you experienced this?
Thanks, Bill
-Long time lurker, first time poster
=================================== Please consider the environment before printing this e-mail Cleveland Clinic is ranked as one of the top hospitals in America by U.S.News & World Report (2014). Visit us online at http://www.clevelandclinic.org for a complete listing of our services, staff and locations. Confidentiality Note: This message is intended for use only by the individual or entity to which it is addressed and may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient or the employee or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. Thank you.
===================================
Please consider the environment before printing this e-mail
Cleveland Clinic is ranked as one of the top hospitals in America by U.S.News & World Report (2014). Visit us online at http://www.clevelandclinic.org for a complete listing of our services, staff and locations.
Confidentiality Note: This message is intended for use only by the individual or entity to which it is addressed and may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient or the employee or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy.
Thank you.
Payne, Richard wrote:
So if you changed the aggr size to 16, you then want to add 4 disks that rg0, then add 4 disks to rg1.
Then you want to run reallocate on the volumes.
Assumnption: this is ONTAP 8.1 or newer. Then: no, you will want to run reallocate -A on the aggregate.
This scanner is impactful, but it does have an effect. Running vol reallocate OTOH is almost always (there are exceptions, as with everything) pointless. If Volumes have lots of inodes, such reallocate jobs never finish anyway
/M
Not sure I follow- before the change, RG1 had 12 and RG2 had 11, then you added 8 disks and now RG1 had 17 and RG2 has 14?
In any case, when you add disks to an aggregate, you should always preview the raid layout it'll build before committing the command.
On Wed, Feb 25, 2015 at 2:15 PM, Woods, Bill WOODSW@ccf.org wrote:
Hello toasters,
A few weeks ago, I had the opportunity to destroy an aggregate in order to consolidate capacity. I started off with two aggregates, each with two RG and both aggr were 600GB drives only the older drives were FC/15K whereas the new aggregate is SAS/10K (I know you can mix RPMs and I’m ok with that). What I’m not ok with is what happened when I attempted to add 8 drives to the new aggregate. The current RG size was 12 and there were 12 drives in the first RG and 11 in the second, I then changed the RG size to 16 and added the drives to the aggregate (expecting to add 4 in each RG). But what really happened is that OnTap added 5 drives to RG1 and created RG2 with the remaining 3 drives.
Support couldn’t find any documentation on this and (I knew this) that it could only be corrected by destroying the aggregate. Have any of you experienced this?
Thanks,
Bill
-Long time lurker, first time poster
===================================
Please consider the environment before printing this e-mail Cleveland Clinic is ranked as one of the top hospitals in America by U.S.News & World Report (2014). Visit us online at http://www.clevelandclinic.org for a complete listing of our services, staff and locations. Confidentiality Note: This message is intended for use only by the individual or entity to which it is addressed and may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient or the employee or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. Thank you.
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Yes. You can specify the disks you want to add and the RG you want to add them to. You cannot undo this, you need to either destroy your aggregate or add more new disks to finish populating your raid groups.
From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Woods, Bill Sent: Wednesday, February 25, 2015 2:15 PM To: toasters@teaparty.net Subject: Adding disks to existing aggregates
Hello toasters,
A few weeks ago, I had the opportunity to destroy an aggregate in order to consolidate capacity. I started off with two aggregates, each with two RG and both aggr were 600GB drives only the older drives were FC/15K whereas the new aggregate is SAS/10K (I know you can mix RPMs and I'm ok with that). What I'm not ok with is what happened when I attempted to add 8 drives to the new aggregate. The current RG size was 12 and there were 12 drives in the first RG and 11 in the second, I then changed the RG size to 16 and added the drives to the aggregate (expecting to add 4 in each RG). But what really happened is that OnTap added 5 drives to RG1 and created RG2 with the remaining 3 drives.
Support couldn't find any documentation on this and (I knew this) that it could only be corrected by destroying the aggregate. Have any of you experienced this?
Thanks, Bill
-Long time lurker, first time poster
=================================== Please consider the environment before printing this e-mail Cleveland Clinic is ranked as one of the top hospitals in America by U.S.News & World Report (2014). Visit us online at http://www.clevelandclinic.org for a complete listing of our services, staff and locations. Confidentiality Note: This message is intended for use only by the individual or entity to which it is addressed and may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient or the employee or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. Thank you.
-
Basil -
Jordan Slingerland -
Michael Bergman -
Payne, Richard -
Sebastian Goetze -
tmac -
Woods, Bill