I'm a bit surprised to see the performance penalty, small though it may
be,
be this way around. I would have assumed that spreading the disc I/O load across controllers as much as possible would be the right thing to do. Can someone explain this seemingly counter-intuitive result in more detail?
I'm just guessing again here, but since the manul confirms that the penalty only occurs on writes, then my guess is that since WAFL writes out a whole stripe at a time, and since you pretty much have to wait for a write to complete when you make it, if your raid group is across two controllers then your stripe is across two controllers and thus you have two bus transfers across the PCI bus and two waits for successful completion, during which time the filer can't do much else.
Note that when they say 10% write penalty, they probably mean as measured by SpecSFS, during heavy loads. It doesn't mean that every time you save a file from your Unix or PC client, it'll take 10% longer. The filer will still return right away; it just slows the filer down a little when it flushes the NVRAM log. I suspect for most environments who only have writes every 5-10 seconds, it wouldn't be a very noticeable hit to performance.
Bruce