I have added mixed sized disks into the same volume, but different raid groups using this method *many* times in the field - it works like a charm. You can bet my life on it too. :-)
The parity disk functionality has been there for a long time. It worked that way when I put the just released 18 GB drives into a 9 GB drive system way back when......
Graham
-----Original Message----- From: Sam Rafter [mailto:rafter@netapp.com] Sent: Wednesday, April 25, 2001 9:36 PM To: toasters@mathworks.com Subject: Re: Adding a 36gb shelf to F740 with 18gb shelves
A funny thing happened to me on the way to the lab. I went to ask the lab manager if I could do some vol destroys on a few filers to get some mixed disk sizes. Our lab manager, Jeff Mohler, is a reformed PSE, and he'd done the very experiment I was about to conduct at customer sites, many times.
Rest assured, you don't need to create a new volume.
Look at your sysconfig -r output presently. Since everything's in one volume, the last raid group listed (raid group 1, if you have two raid groups like you say you do) is the raid group that's subject to change in size. This is controlled by the vol option raidsize. Set the raid size to whatever this raid group size is presently by 'vol options <volname> raidsize <n>'. Add your disks. If the number of disks you are adding is greater than <n>, add some number of 36G drives less than n, and change the vol option to the number of 36G disks you intend to add.
I had fears that the raid group size would be stored in the raid group by some pernicious machinations, but Jeff assuaged any such trepidations. I'll bet Jeff's life on it. If you have one 18G spare, and one 36G spare, ONTAP now has the smarts to pick the right sized spare. If you are using mixed raid group sizes, I recommend you run ONTAP 6.x to take advantage of the spacemap, and avoid some unpleasant endgame scenarios if you do manage to fill up your disks.
-- Sam Rafter Escalations Jerk Network Appliance rafter@netapp.com
Sam Rafter wrote: > > This used to be the case. Back in the sad days of bug 13880. > > http://now.netapp.com/NOW/cgi-bin/bol?Type=Detail&Display=13880 > > This issue is resolved with the whiz-bang spacemap as implemented in 6.0 > and later, where we have a much better idea of where all the space is, > and won't paint ourselves into a corner if we don't have to. > > As to the parity disk question, to quote our own dogfood: > > http://now.netapp.com/NOW/knowledge/docs/ontap/rel601r3/html/sag/disks4.htm > > "Parity disk determination: > If an added disk is larger than the existing parity disk, the added disk > becomes the parity disk, and the former parity disk becomes a data > disk." > > We'll bring Justin back to the lab for reprogramming. > > Now, a reasonable follow up question would be: can I make a new raid > group out of all 36G drives, and not waste any 36G drives as parity > disks on 18G raid groups? My understanding is that, if you change the > volume's raid group size to whatever the size of the 'last' 18G disk > raid group is, we'll make a new raid group out of the 36G drives. I'm > going to test this now. > > -- > Sam Rafter Escalations Jerk > Network Appliance rafter@netapp.com > > "Sullivan, Justin" wrote: > > > > Yes, you're going to have a problem.. :( > > > > If you add 36GB drives to an existing RAID group of 18GB disks, the 36GB disks will be reported and used as 18GB drives. > > > > Once a RAID group is established with a parity drive of xxGB, any disks added to that RAID group will only be used to the same capacity (xxGB) as the parity drive. > > > > If you reverse the situation, and add 18GB drives to an existing 36GB drive RAID group, all drives will be used to their full capacity, but you will run into performance troubles as the volume fills up and the system is forced to write only to the larger disks, thereby reducing the number of physical spindles working for you. > > > > You will definitely want to create a new RAID group (or a new volume) for your 36GB drives to take advantage of the higher capacity. > > > > --- > > Justin Sullivan > > Network Appliance, Technical Support Engineer > > Network Appliance Certified Associate > > justins@netapp.com > > Get answers NOW! - NetApp On the Web - http://now.netapp.com > > Tech Support Hotline: 1-888-4NETAPP > > > > -----Original Message----- > > From: Jeff Bryer [mailto:bryer@sfu.ca] > > Sent: Wednesday, April 25, 2001 6:26 PM > > To: toasters@mathworks.com > > Subject: Adding a 36gb shelf to F740 with 18gb shelves > > > > We have an F740 with 3 shelves of 18GB disks. It's config'd > > as two volumes (two raid sets). We've almost outgrown the > > capacity (we're down to one hot spare and a free disk). > > > > We want to add a new disk shelf to the system, but we're wondering > > if we're going to lose anything by going to a shelf of 36gb disks. > > We don't have any intention of adding new volumes (or new raid > > sets), just growing the existing volumes. > > > > Can we add a shelf of 36gb disks and then switch our 2 existing 18gb parity > > disks over to 36gb disks? We'd keep a 36gb disk as a hot spare and > > use all of the 18gb disks in the volumes (including the 2 freed up > > parity disks and the freed up hot spare). Are we going to run > > into problems with this? Or do we need to keep an 18gb disk around > > as a hot spare as well (ie when an 18gb disks dies and uses a 36gb > > disks as a hot spare, can we force it to go back to the 18gb disk > > once the faulty 18gb disk has been replaced?). > > > > Are the 36gb disks going to be used at full capacity in this situation > > (when mixed with the 18gb)? Or is there a better way to do this? > > > > Thanks, > > > > -- > > Jeff Bryer bryer@sfu.ca > > Systems Administrator (604) 291-4935 > > Academic Computing, Simon Fraser University
I second this, we're doing this right now:
RAID Disk HA.ID HA SHELF BAY CHAN Used (MB/blks) Phys (MB/blks) --------- ----- ------------ ---- -------------- -------------- parity 8a.13 8a 1 5 8600/17612800 8683/17783240 data 7b.14 7b 1 6 4000/8192000 4095/8388312 data 7b.6 7b 0 6 4000/8192000 4095/8388312 data 7b.12 7b 1 4 4000/8192000 4095/8388312 data 8a.9 8a 1 1 8600/17612800 8683/17783240 data 8a.12 8a 1 4 8600/17612800 8683/17783240 data 8a.10 8a 1 2 8600/17612800 8683/17783240 data 8a.4 8a 0 5 8600/17612800 8683/17783240 ... etc, etc...
There were no performance penalties that we could measure.
One HUGE recommendation though - *always* keep at least one spare of each disk size in the filer. If one of the smaller disks crashes, ONTAP will earn a gold star and rebuild on the smallest drive that can hold the data. If thats the same size as the failed drive - great! If it is larger (i.e., failed 4GB drive and only 9GB spares left), then it will only write to a region the size of the smaller disk and waste the remaining space.
The only fix to this is the fail the larger drive that just got rebuilt upon out, add a spare of the correct (smaller) size and let ONTAP rebuild there, then put in a *new, fresh* drive of the larger size back in. A big pain in the butt.
So learn from us and keep your vol status -r looking like this:
Spare disks
RAID Disk HA.ID HA SHELF BAY CHAN Used (MB/blks) Phys (MB/blks) --------- ----- ------------ ---- -------------- -------------- spare 8a.0 8a 0 1 0 8683/17783240 spare 8b.14 8b 1 6 0 4095/8388312
BTW, Mr. Mohler: I know that the only swag on Sam's desk is of the single-malt variety. I'll bet Sam's life on it! =)
-- Jeff
On Thu, Apr 26, 2001 at 08:28:14AM -0700, Knight, Graham wrote:
I have added mixed sized disks into the same volume, but different raid groups using this method *many* times in the field - it works like a charm. You can bet my life on it too. :-)
The parity disk functionality has been there for a long time. It worked that way when I put the just released 18 GB drives into a 9 GB drive system way back when......
Graham
-----Original Message----- From: Sam Rafter [mailto:rafter@netapp.com] Sent: Wednesday, April 25, 2001 9:36 PM To: toasters@mathworks.com Subject: Re: Adding a 36gb shelf to F740 with 18gb shelves A funny thing happened to me on the way to the lab. I went to ask the lab manager if I could do some vol destroys on a few filers to get some mixed disk sizes. Our lab manager, Jeff Mohler, is a reformed PSE, and he'd done the very experiment I was about to conduct at customer sites, many times. Rest assured, you don't need to create a new volume. Look at your sysconfig -r output presently. Since everything's in one volume, the last raid group listed (raid group 1, if you have two raid groups like you say you do) is the raid group that's subject to change in size. This is controlled by the vol option raidsize. Set the raid size to whatever this raid group size is presently by 'vol options <volname> raidsize <n>'. Add your disks. If the number of disks you are adding is greater than <n>, add some number of 36G drives less than n, and change the vol option to the number of 36G disks you intend to add. I had fears that the raid group size would be stored in the raid group by some pernicious machinations, but Jeff assuaged any such trepidations. I'll bet Jeff's life on it. If you have one 18G spare, and one 36G spare, ONTAP now has the smarts to pick the right sized spare. If you are using mixed raid group sizes, I recommend you run ONTAP 6.x to take advantage of the spacemap, and avoid some unpleasant endgame scenarios if you do manage to fill up your disks. -- Sam Rafter Escalations Jerk Network Appliance rafter@netapp.com Sam Rafter wrote: > > This used to be the case. Back in the sad days of bug 13880. > > http://now.netapp.com/NOW/cgi-bin/bol?Type=Detail&Display=13880 > > This issue is resolved with the whiz-bang spacemap as implemented in 6.0 > and later, where we have a much better idea of where all the space is, > and won't paint ourselves into a corner if we don't have to. > > As to the parity disk question, to quote our own dogfood: > > http://now.netapp.com/NOW/knowledge/docs/ontap/rel601r3/html/sag/disks4.htm > > "Parity disk determination: > If an added disk is larger than the existing parity disk, the added disk > becomes the parity disk, and the former parity disk becomes a data > disk." > > We'll bring Justin back to the lab for reprogramming. > > Now, a reasonable follow up question would be: can I make a new raid > group out of all 36G drives, and not waste any 36G drives as parity > disks on 18G raid groups? My understanding is that, if you change the > volume's raid group size to whatever the size of the 'last' 18G disk > raid group is, we'll make a new raid group out of the 36G drives. I'm > going to test this now. > > -- > Sam Rafter Escalations Jerk > Network Appliance rafter@netapp.com > > "Sullivan, Justin" wrote: > > > > Yes, you're going to have a problem.. :( > > > > If you add 36GB drives to an existing RAID group of 18GB disks, the 36GB disks will be reported and used as 18GB drives. > > > > Once a RAID group is established with a parity drive of xxGB, any disks added to that RAID group will only be used to the same capacity (xxGB) as the parity drive. > > > > If you reverse the situation, and add 18GB drives to an existing 36GB drive RAID group, all drives will be used to their full capacity, but you will run into performance troubles as the volume fills up and the system is forced to write only to the larger disks, thereby reducing the number of physical spindles working for you. > > > > You will definitely want to create a new RAID group (or a new volume) for your 36GB drives to take advantage of the higher capacity. > > > > --- > > Justin Sullivan > > Network Appliance, Technical Support Engineer > > Network Appliance Certified Associate > > justins@netapp.com > > Get answers NOW! - NetApp On the Web - http://now.netapp.com > > Tech Support Hotline: 1-888-4NETAPP > > > > -----Original Message----- > > From: Jeff Bryer [mailto:bryer@sfu.ca] > > Sent: Wednesday, April 25, 2001 6:26 PM > > To: toasters@mathworks.com > > Subject: Adding a 36gb shelf to F740 with 18gb shelves > > > > We have an F740 with 3 shelves of 18GB disks. It's config'd > > as two volumes (two raid sets). We've almost outgrown the > > capacity (we're down to one hot spare and a free disk). > > > > We want to add a new disk shelf to the system, but we're wondering > > if we're going to lose anything by going to a shelf of 36gb disks. > > We don't have any intention of adding new volumes (or new raid > > sets), just growing the existing volumes. > > > > Can we add a shelf of 36gb disks and then switch our 2 existing 18gb parity > > disks over to 36gb disks? We'd keep a 36gb disk as a hot spare and > > use all of the 18gb disks in the volumes (including the 2 freed up > > parity disks and the freed up hot spare). Are we going to run > > into problems with this? Or do we need to keep an 18gb disk around > > as a hot spare as well (ie when an 18gb disks dies and uses a 36gb > > disks as a hot spare, can we force it to go back to the 18gb disk > > once the faulty 18gb disk has been replaced?). > > > > Are the 36gb disks going to be used at full capacity in this situation > > (when mixed with the 18gb)? Or is there a better way to do this? > > > > Thanks, > > > > -- > > Jeff Bryer bryer@sfu.ca > > Systems Administrator (604) 291-4935 > > Academic Computing, Simon Fraser University