Thanks, everyone. I have manually added a spare on each of the nodes to their aggregates, leaving me with a spare left for each one. Yes, the disks are all the same size.
Also, yes, one of the shelves is not completely full. This has happened with the last few groups of shelves we have gotten, leaving us with an odd number. Not totally sure why it happens, but I'm guessing they just give us however many disks it takes to reach the amount of raw space we ordered. They probably don't care if that makes the shelves even or not.
-----Original Message----- From: Chris Muellner [mailto:chrism@mochadata.com] Sent: Tuesday, October 28, 2008 5:01 PM To: Brian.Beaird@cat.com; toasters@mathworks.com Subject: RE: Nodes Sharing Spares
Unfortunately there isn't a way to share spares among the controllers. Each node requires its own set of spare drives for the disks that it owns. You'll want to have at least one spare per drive type, and one of those spares per 2-3 shelves of that drive type depending on who you talk to. For 21 total disks there would be no reason to have 4 spares though.
If they're all the same drive type and split between the two heads then you would be find with one spare of that drive type per controller. If some are SAS or FC and some are ATA then you would want to have the node that owns the SAS/FC drives to have one spare of that type and the head that owns the ATA to have one spare of that type. If, however, you have them split evenly between the nodes then each node would need to have a spare of each type.
Can I ask how you have 21 total physical disks? Is there a half-filled shelf in the system?
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Brian.Beaird@cat.com Sent: Tuesday, October 28, 2008 3:16 PM To: toasters@mathworks.com Subject: Nodes Sharing Spares
We are using the IBM n-series filers with our aggregates using dual-parity. Our filer is a dual-node controller, so each node independently owns different disks from the storage shelves. My question deals with how spare disks are allocated. With how it's currently set up, each node has to reserve two disks that it owns as spares to support dual parity. I don't know that we necessarily need four spares for such a relatively small number of physical disks (21 total). It seems pretty wasteful to have all that space just sitting there, especially when the chances of more than two disks failing before we can get replacements in are pretty slim.
Is there some way to just let the nodes share a couple spares so that either one can grab them if failures occur in one of their owned disks? It would be great if I could free up one or two of those spares for actual use. I know there's a way to force spare disksk into aggregates, but I don't want to do this if that means one node won't have any spares to use.
Thanks, Brian Beaird