We have a F740 filer with two 18 gig shelves. Each shelf is dedicated to a volume.We have reserved disk on each shelf as a hotspare for that particular volume. So in total we have two hot spares.

Last week one of the disks failed on the first volume and failed disk was recontructed on the spare which was physically present on second shelf. So we ended up with the volume which transgresses over to another shelf. Our goal was to contain each volume to a shelf of its own.

First,is there a way to contain the volumes with hot spares? Secondly how do I make the first volume give up the disk on second shelf and recontruct it on spare which is physically present on first shelf.

I hope I made myself clear.

Thanks,

Bhavnesh

I just got back from "NetApp 202 Advanced System Admin & Troubleshooting" class. They went over a similar scenario. They had two volumes, one on each of two controllers. (they said that there is 10 percent performance hit, when drives in an volume are on different controllers). For example, if they had two volumes, each with six drives, and the all of the drives for the first volume were on the first controller and all of the drives for the second volume were on the second controller. And then a drive failed in the second volume, and rebuilt on a spare on the first controller. There would be a 10 percent performance hit, every time you accessed the second volume.

This is what they said to do to:

Replace the failed drive in the second volume. Then pull out all of the spares, including the one that was used to rebuild the second volume. This will force the volume to rebuild on the new drive.

Bhavnesh Makin wrote:

We have a F740 filer with two 18 gig shelves. Each shelf is dedicated to a volume.We have reserved disk on each shelf as a hotspare for that particular volume. So in total we have two hot spares.

Last week one of the disks failed on the first volume and failed disk was recontructed on the spare which was physically present on second shelf. So we ended up with the volume which transgresses over to another shelf. Our goal was to contain each volume to a shelf of its own.

First,is there a way to contain the volumes with hot spares? Secondly how do I make the first volume give up the disk on second shelf and recontruct it on spare which is physically present on first shelf.

I hope I made myself clear.

Thanks,

Bhavnesh

This is what the book says:

"-Write performance will suffer when a RAID group spans more than one FC-AL or SCSI adapter. -Keep all drives assigned to a RAID group on the same adapter whenever posible. -There is an approximate 10% decrease in write performance when the filer attempts to write to a RAID group spanning two adapters. This is due to inherent limitations in the PCI bus."

Bruce Sterling Woodcock wrote:

Unless I misunderstand some of the bus issues involved, the 10% performance hit would only occur on writes, and since writes are grouped and it is unlikely that you are writing to the drives ever second, you'll never notice it.

Reads come off individual disks as needed, so it doesn't matter which controller they are on. (I guess if you needed to read a whole stripe at once there's a potential hit, but I'm not sure that necessary.) The point is, with two volumes you're using both controllers anyway.

Bruce

I'm a bit surprised to see the performance penalty, small though it may

be,

be this way around. I would have assumed that spreading the disc I/O load across controllers as much as possible would be the right thing to do. Can someone explain this seemingly counter-intuitive result in more detail?

I'm just guessing again here, but since the manul confirms that the penalty only occurs on writes, then my guess is that since WAFL writes out a whole stripe at a time, and since you pretty much have to wait for a write to complete when you make it, if your raid group is across two controllers then your stripe is across two controllers and thus you have two bus transfers across the PCI bus and two waits for successful completion, during which time the filer can't do much else.

Note that when they say 10% write penalty, they probably mean as measured by SpecSFS, during heavy loads. It doesn't mean that every time you save a file from your Unix or PC client, it'll take 10% longer. The filer will still return right away; it just slows the filer down a little when it flushes the NVRAM log. I suspect for most environments who only have writes every 5-10 seconds, it wouldn't be a very noticeable hit to performance.

Bruce