Douglas,
I think you are confusing "adapter" with "disk shelf". Unless you have 2 FCAL adapters in your filer, and have connected each shelf to its own adapter, you are not spanning adapters and thus not violating what the manual says. I have many filers with their full complement of disk shelves connecting to the 1 FCAL adapter on the filer head. I have RAID groups that span shelves as will happen when disk spare out. There is no performance penalty in this scenario.
Sam Schorr Homestead.com ph: (650) 549-3152 fax: (650) 364-7329 sschorr@homestead-inc.com http://www.homestead.com
Homestead: It's a new world. Make your place.
-----Original Message----- From: Douglas Ritschel [mailto:Douglas.Ritschel@fnc.fujitsu.com] Sent: Monday, July 10, 2000 12:20 PM To: Bruce Sterling Woodcock Cc: toasters@mathworks.com Subject: Re: volume/shelf containment
This is what the book says:
"-Write performance will suffer when a RAID group spans more than one FC-AL or SCSI adapter. -Keep all drives assigned to a RAID group on the same adapter whenever posible. -There is an approximate 10% decrease in write performance when the filer attempts to write to a RAID group spanning two adapters. This is due to inherent limitations in the PCI bus."
Bruce Sterling Woodcock wrote:
Unless I misunderstand some of the bus issues involved, the 10% performance hit would only occur on writes, and since writes are grouped and it is unlikely that you are writing to the drives ever second, you'll never notice it.
Reads come off individual disks as needed, so it doesn't matter which controller they are on. (I guess if you needed to read a whole stripe at once there's a potential hit, but I'm not sure that necessary.) The point is, with two volumes you're using both controllers anyway.
Bruce
In my original note, I was using the example given in class (which had multiple controllers). I know the difference between shelves and controllers. I just thought that the 10 percent performance hit, when crossing controllers would be useful information, I should have left it out.
The main point was that there is no way to tell the filer to build on a specific spare. And the only way, according to the NetApp class that I took, (without shutting down and shuffling drives), is to replace the failed drive, and remove all of the spares, including the one that the array rebuilt on. Without any spares in the system, the array will rebuild on the new drive.
Sam Schorr wrote:
Douglas,
I think you are confusing "adapter" with "disk shelf". Unless you have 2 FCAL adapters in your filer, and have connected each shelf to its own adapter, you are not spanning adapters and thus not violating what the manual says. I have many filers with their full complement of disk shelves connecting to the 1 FCAL adapter on the filer head. I have RAID groups that span shelves as will happen when disk spare out. There is no performance penalty in this scenario.
Sam Schorr Homestead.com ph: (650) 549-3152 fax: (650) 364-7329 sschorr@homestead-inc.com http://www.homestead.com
Homestead: It's a new world. Make your place.
-----Original Message----- From: Douglas Ritschel [mailto:Douglas.Ritschel@fnc.fujitsu.com] Sent: Monday, July 10, 2000 12:20 PM To: Bruce Sterling Woodcock Cc: toasters@mathworks.com Subject: Re: volume/shelf containment
This is what the book says:
"-Write performance will suffer when a RAID group spans more than one FC-AL or SCSI adapter. -Keep all drives assigned to a RAID group on the same adapter whenever posible. -There is an approximate 10% decrease in write performance when the filer attempts to write to a RAID group spanning two adapters. This is due to inherent limitations in the PCI bus."
Bruce Sterling Woodcock wrote:
Unless I misunderstand some of the bus issues involved, the 10% performance hit would only occur on writes, and since writes are grouped and it is unlikely that you are writing to the drives ever second, you'll never notice it.
Reads come off individual disks as needed, so it doesn't matter which controller they are on. (I guess if you needed to read a whole stripe at once there's a potential hit, but I'm not sure that necessary.) The point is, with two volumes you're using both controllers anyway.
Bruce
-- Douglas Ritschel Fujitsu Network Communications Two Blue Hill Plaza, Sixth Floor P.O. Box 1609 Pearl River, NY 10965 Ph: 914.731.2101 Fx: 914-731-2011 Douglas.Ritschel@fnc.fujitsu.com
Wouldn't it be at least slightly risky to pull all your spares plus one of your data drives for the duration of a rebuild?
Frank
--On 07/10/00 16:57:17 -0400 Douglas Ritschel Douglas.Ritschel@fnc.fujitsu.com wrote:
The main point was that there is no way to tell the filer to build on a specific spare. And the only way, according to the NetApp class that I took, (without shutting down and shuffling drives), is to replace the failed drive, and remove all of the spares, including the one that the array rebuilt on. Without any spares in the system, the array will rebuild on the new drive.
Frank Smith fsmith@hoovers.com Systems Administrator Voice: 512-374-4673 Hoover's Online Fax: 512-374-4501
Yes!
I wouldn't do that unless someone held a gun to my head (imagine _that_ scenerio!).
what you (Douglas) could do is buy a bigger drive (36GB if you have 18's) and plop that in as a spare, then leave only one 18GB spare and the 36GB spare and it should build on the 18GB first (if the rest of your disks are 18 of course) then if you have another loss, it'll use the 36GB.
I believe the instructors intent was probably to say that it is possible, but you shouldn't do it...
-corris
On Mon, 10 Jul 2000, Frank Smith wrote:
Date: Mon, 10 Jul 2000 18:10:10 -0500 From: Frank Smith fsmith@hoovers.com To: toasters@mathworks.com Subject: Re: volume/shelf containment
Wouldn't it be at least slightly risky to pull all your spares plus one of your data drives for the duration of a rebuild?
Frank
--On 07/10/00 16:57:17 -0400 Douglas Ritschel Douglas.Ritschel@fnc.fujitsu.com wrote:
The main point was that there is no way to tell the filer to build on a specific spare. And the only way, according to the NetApp class that I took, (without shutting down and shuffling drives), is to replace the failed drive, and remove all of the spares, including the one that the array rebuilt on. Without any spares in the system, the array will rebuild on the new drive.
Frank Smith fsmith@hoovers.com Systems Administrator Voice: 512-374-4673 Hoover's Online Fax: 512-374-4501
what you (Douglas) could do is buy a bigger drive (36GB if you have 18's) and plop that in as a spare, then leave only one 18GB spare and the 36GB spare and it should build on the 18GB first (if the rest of your disks are 18 of course) then if you have another loss, it'll use the 36GB.
Unless the failed disk is the parity, in which case the 36gb will be used. Putting only 1 of a larger disk is always a dangerous thing, as you're essentially running w/o a spare.
Further, if you rebuild a 18 onto a 36, you _don't_ get a bigger volume -- you get a more expensive 18gb disk (which you'd probably have to relabel to get the filer to use all 36gb of again).
Some features to support rolling upgrades of disks would be damn smooth though. (proactively fail a disk onto a bigger one, which grows the volume and rebuilds).. Makes an upgrade path to $BIGDISK[$#BIGDISK+1] quite attractive.
..kg..
----- Original Message ----- From: "kevin graham" kgraham@dotnetdotcom.org To: "Corris Randall" corris@acc.am.ericsson.se Cc: toasters@mathworks.com Sent: Tuesday, July 11, 2000 11:13 AM Subject: Re: volume/shelf containment
what you (Douglas) could do is buy a bigger drive (36GB if you have 18's) and plop that in as a spare, then leave only one 18GB spare and
the
36GB spare and it should build on the 18GB first (if the rest of your disks are 18 of course) then if you have another loss, it'll use the
36GB.
Unless the failed disk is the parity, in which case the 36gb will be used.
Why would it do that, when the parity is only 36GB? We're talking about what to do during a failure, not adding a new disk. If it fails over to the 36GB drive when an 18 is available, I would consider that a bug.
Putting only 1 of a larger disk is always a dangerous thing, as you're essentially running w/o a spare.
I don't follow what you mean here. He's talking about only one 36GB drive and having it be a "second spare". (Presumably this is safer than having only one global spare available, since you want it to use the 36GB in case the other raid group fails before you can replace it.) Of course, the other volume could still fail and build on the 18GB disk on the other shelf.
Further, if you rebuild a 18 onto a 36, you _don't_ get a bigger volume -- you get a more expensive 18gb disk (which you'd probably have to relabel to get the filer to use all 36gb of again).
AFAIK, you don't have to relabel to do this, although you would need to go through a raid fail to replace it with an 18.
Some features to support rolling upgrades of disks would be damn smooth though. (proactively fail a disk onto a bigger one, which grows the volume and rebuilds).. Makes an upgrade path to $BIGDISK[$#BIGDISK+1] quite attractive.
That's already supported for the parity drive (so you can add bigger disks in the first place) but doing it for a data drive would be a little trickier. (If you had a situation where the smaller parity failed and *only* a bigger drive is available, it would be nice if it did this too, per your concern in the first paragraph... I don't know if the filer actually does this, though?)
Bruce
Unless the failed disk is the parity, in which case the 36gb will be used.
[...]
If it fails over to the 36GB drive when an 18 is available, I would consider that a bug.
The largest spare available will always be used to rebuild a parity disk, regardless of its original size (from the SA Guide). Just closed the PDF, but it was under 'managing disks'.
Putting only 1 of a larger disk is always a dangerous thing, as you're essentially running w/o a spare.
I don't follow what you mean here. He's talking about only one 36GB drive and having it be a "second spare". (Presumably this is safer than having only one global spare available, since you want it to use the 36GB in case the other raid group fails before you can replace it.)
Ah, I must have gotten lost in the thread.. I had thought this was an attempt to start adding 36's and force a fail to move to the new disk.
..kg..
----- Original Message ----- From: "kevin graham" kgraham@dotnetdotcom.org To: "Bruce Sterling Woodcock" sirbruce@ix.netcom.com Cc: "Corris Randall" corris@acc.am.ericsson.se; toasters@mathworks.com Sent: Tuesday, July 11, 2000 12:36 PM Subject: Re: volume/shelf containment
Unless the failed disk is the parity, in which case the 36gb will be used.
[...]
If it fails over to the 36GB drive when an 18 is available, I would consider that a bug.
The largest spare available will always be used to rebuild a parity disk, regardless of its original size (from the SA Guide). Just closed the PDF, but it was under 'managing disks'.
Hmm. I'd consider that a bug, even if it "rebuilds and resizes to the new size" as it does when you add a new drive. If I have a legacy raid group of 18GB drives, and a new group of 36GB drives, I want the parity drive from the smaller RAID group to chosen the smaller global parity spare.
Any comments from Netapp personnel?
Putting only 1 of a larger disk is always a dangerous thing, as you're essentially running w/o a spare.
I don't follow what you mean here. He's talking about only one 36GB drive and having it be a "second spare". (Presumably this is safer than having only one global spare available, since you want it to use the
36GB
in case the other raid group fails before you can replace it.)
Ah, I must have gotten lost in the thread.. I had thought this was an attempt to start adding 36's and force a fail to move to the new disk.
Or maybe I was the lost one... we've discussed a couple of different things...
Bruce
-
Bruce Sterling Woodcock -
Corris Randall -
Douglas Ritschel -
Frank Smith -
kevin graham -
Sam Schorr