I just got in an F630 with 5.0.2 and four fully-loaded differential shelves. Yowza. It showed up with one volume, vol0, two RAID groups (one for each pair of shelves, I guess), and only two disks in use, in vol0.
I want to live dangerously, with one big volume and one RAID group for it. After all, that's how my other F630's running 4.x are configured, and it's worked fine for a long time.
So naturally I did something stooopid. I (a) added all the disks, except for a spare, to vol0, (b) set the RAID group size of vol0 to 26, and then (c) noticed in the 5.x documentation that setting the RAID group size AFTER the fact don't do squat. :-( I might have had a prayer if I'd done (b) then (a).
Here's what I have now:
pacer> sysconfig -r Volume vol0
RAID group 0
RAID Disk HA.Disk_ID Used (MB/blks) Phys (MB/blks) --------- ---------- -------------- -------------- parity 8b.13 8600/17612800 8683/17783240 data 8b.14 8600/17612800 8683/17783240 data 8b.1 8600/17612800 8683/17783240 data 8a.1 8600/17612800 8683/17783240 data 8a.2 8600/17612800 8683/17783240 data 8a.3 8600/17612800 8683/17783240 data 8a.4 8600/17612800 8683/17783240 data 8a.5 8600/17612800 8683/17783240 data 8a.8 8600/17612800 8683/17783240 data 8a.9 8600/17612800 8683/17783240 data 8a.10 8600/17612800 8683/17783240 data 8a.11 8600/17612800 8683/17783240 data 8a.12 8600/17612800 8683/17783240 data 8a.13 8600/17612800 8683/17783240
RAID group 1
RAID Disk HA.Disk_ID Used (MB/blks) Phys (MB/blks) --------- ---------- -------------- -------------- parity 8b.2 8600/17612800 8683/17783240 data 8a.14 8600/17612800 8683/17783240 data 8b.3 8600/17612800 8683/17783240 data 8b.4 8600/17612800 8683/17783240 data 8b.5 8600/17612800 8683/17783240 data 8b.8 8600/17612800 8683/17783240 data 8b.9 8600/17612800 8683/17783240 data 8b.10 8600/17612800 8683/17783240 data 8b.11 8600/17612800 8683/17783240 data 8b.12 8600/17612800 8683/17783240
Spare disks
RAID Disk HA.Disk_ID Used (MB/blks) Phys (MB/blks) --------- ---------- -------------- -------------- spare 8b.0 0 8683/17783240 spare 8a.0 0 8683/17783240
And what I want to have is only one RAID group, 0, with one parity and one spare.
It looks like I am going to have to clobber vol0 and start over. Am I right? Anybody have experience with offlining then destroying vol0?
Brian Rice brice@gnac.com
I (a) added all the disks, except for a spare, to vol0, (b) set the RAID group size of vol0 to 26
So at one parity disk per 14 the overhead is 1/14 = 7%, and at one parity disk per 26 the overhead is 3.8%.
So are you dead-set on living dangerously?
Seriously, you actually get a double benefit from having smaller RAID groups. The first benefit, of course, is that the chances of losing two disks out of 14 is less than the chances of losing two disks out of 26.
But in addition, RAID reconstruction is much faster. When you reconstruct a failed drive, you have to read all the data on all the other drives, so if your RAID groups are half as big, then the reconstruction goes twice as fast.
Dave
I (a) added all the disks, except for a spare, to vol0, (b) set the RAID group size of vol0 to 26
So at one parity disk per 14 the overhead is 1/14 = 7%, and at one parity disk per 26 the overhead is 3.8%.
So are you dead-set on living dangerously?
Seriously, you actually get a double benefit from having smaller RAID groups. The first benefit, of course, is that the chances of losing two disks out of 14 is less than the chances of losing two disks out of 26.
But in addition, RAID reconstruction is much faster. When you reconstruct a failed drive, you have to read all the data on all the other drives, so if your RAID groups are half as big, then the reconstruction goes twice as fast.
Which reduces the exposure to a double-disk failure (because the system is down one disk for a shorter window) as well as reducing the time during which performance is impaired by the reconstruction.
These issues are documented in excruciating detail in TR-3027, which you can find at http://www.netapp.com/technology/level3/3027.html.
-- Karl Swartz - Technical Marketing Engineer Network Appliance Work: kls@netapp.com http://www.netapp.com/ Home: kls@chicago.com http://www.chicago.com/~kls/
On Fri, 4 Sep 1998, Dave Hitz wrote:
Seriously, you actually get a double benefit from having smaller RAID groups. The first benefit, of course, is that the chances of losing two disks out of 14 is less than the chances of losing two disks out of 26.
And furthermore, is there any reason that those two raid groups should be equal size? Does the OS distribute the load evenly amongst raid groups, or does it weigh them based on disks, etc?
On Fri, 4 Sep 1998, Dave Hitz wrote:
Seriously, you actually get a double benefit from having smaller RAID groups. The first benefit, of course, is that the chances of losing two disks out of 14 is less than the chances of losing two disks out of 26.
And furthermore, is there any reason that those two raid groups should be equal size? Does the OS distribute the load evenly amongst raid groups, or does it weigh them based on disks, etc?
We experimented with writing to just one RAID group at a time, but that lead to performance problems when one of the RAID groups was small, so now we distribute over all RAID groups in a volume to maintain performance.
Dave
On Fri, Sep 04, 1998 at 03:16:27PM -0700, Dave Hitz wrote:
On Fri, 4 Sep 1998, Dave Hitz wrote:
Seriously, you actually get a double benefit from having smaller RAID groups. The first benefit, of course, is that the chances of losing two disks out of 14 is less than the chances of losing two disks out of 26.
And furthermore, is there any reason that those two raid groups should be equal size? Does the OS distribute the load evenly amongst raid groups, or does it weigh them based on disks, etc?
We experimented with writing to just one RAID group at a time, but that lead to performance problems when one of the RAID groups was small, so now we distribute over all RAID groups in a volume to maintain performance.
So, with 2 raid groups...it is all still one filesystem?
-
Brian Rice -
hitz＠netapp.com -
John K. Edwards -
kls＠netapp.com -
Matt Stein -
Mike Horwath