They (Netapp) spent a fair amount time in the Netapp 202 class teaching about how it is best to keep a RAID group on one controller. If you have a single RAID group that spans more than one FC-AL interface, it is called a fractured RAID group, and there was even a lab in the class where they went through how to correct a fractured RAID group and put the disks back on one FC-AL controller.
It looks like the configuration that was used in the benchmark was a single volume that had 2 RAID groups (RAID group 0, and RAID group 1, presumably), and each of these RAID groups was cabled up to separate interfaces. I'm guessing that this configuration would yield a filesystem that is striped across both FC-AL interfaces. Since the RAID calculations are done at the RAID group level rather than at the volume level it makes sense that more than one RAID calculation can be done in parallel. Based on that assumption, splitting a volume across 2 controllers (while preventing RAID group fracturing) should improve performance.
Rick Hulsey Southwest Airlines 2425 Wyman Mail Stop 4DC Dallas, Tx., 75235 rick.hulsey@wnco.com (214) 792-7188 Office (972) 880-6882 Cell (800) 915-3747 Pager
Graydon Dodson grdodson@lexmark.com 10/09/00 04:22PM >>>
I also had a 760 survive doing 17,000 nfs ops/second once. ...
When running compiles, it is not unusual for us to see numbers over 22,000 on the op-panel of our 760. Anyone know where the top end of an 840 is? Support has been unable to answer that one for us.
So in our move to an 840 I want to configure it's volume for maximum performance. The ground rules are two FC-AL adapters, seven full trays of 18G drives. All one volume is very desirable.
Now the SPECsfs97 benchmark on the 840 states:
* The F1 filesystem was composed of two RAID groups, each containing 17 data disks and one parity disk. Two spare disks were present
* The F1 filesystem was striped across both disk controllers
Since I have to assume that NetApp would use a high performance architecture for the benchmark I am guessing that this is the way to go, but there is quite a bit of detail missing.
Is it a good or bad idea to split a raid group over two FC-AL interfaces?
What does "striped across both disk controllers" mean?
If there are spare disks (same size) on both FC-AL interfaces and a disk fails Which one is used to rebuild? (Same FC-AL or random choice)
Is it a good or bad idea to split a volume over multiple FC-AL interfaces?
The need for speed
Graydon Dodson (606) 232-6483 grdodson@lexmark.com Lexmark International Inc.
On Tue, 10 Oct 2000, Rick Hulsey wrote:
It looks like the configuration that was used in the benchmark was a single volume that had 2 RAID groups (RAID group 0, and RAID group 1, presumably), and each of these RAID groups was cabled up to separate interfaces. I'm guessing that this configuration would yield a filesystem that is striped across both FC-AL interfaces. Since the RAID calculations are done at the RAID group level rather than at the volume level it makes sense that more than one RAID calculation can be done in parallel.
This triggered a question in my mind... <flipping through my 202 class notes...> Here we go. <ahem>
"A reading from the book of NetApp, Chapter "Performance Tuning," Page 7, Paragraph I of the April 2000 notes..." [0]
Hitz sayeth (I assume):
"WAFL writes data in a round-robin fashion to the RAID groups in a volume. Data is written to one RAID group at a time, one per consistency point. Because of the write allocation policy the filer will attempt to distribute writes evenly between RAID groups regardless of size."
My questions then are:
o Is it even a good idea to have more than one RAID group per volume?
Say I fill up a volume on a 14-disk RAID group (12d+1p+1hs), and then I add two more shelves (13d+1p) in another RAID group to expand this volume. (This gets rid of the RAID group size imbalance that will definitely result in poorer performance, which is the next subject discussed after the quote above.) Say I continue to add data to the volume, and as is likely, the last added data is probably the most often used. Will I have a "hot spot" in the second RAID group with the consistency points, since I'll be doing lots of reads and writes to the new data and very little to the old data?
o If the next consistency point is the one that is going to update RAID group #2, where does the data go that needs to be written to RAID group #1?
o Can the filer get into a state of hurryup!...wait...hurryup!....wait in the above example, or in the example of severely imbalanced RAID group sizes in the same volume? Or, do the consistency points get written out sooner than every 10 seconds even if half the NVRAM isn't filled, in order to "catch up" with the more active RAID group?
I'm not sure if these are stupid questions or so esoteric as to be practically meaningless. Nevertheless, I'm curious.
[0] Another Monty Python veiled reference. For some reason, I'm on a kick lately...
And Hitz spake, saying, 'First shalt thou take out the Holy Failed Disk. Then, shalt thou count to three, no more, no less. Three shalt be the number thou shalt count, and the number of the counting shall be three. Four shalt thou not count, nor either count thou two, excepting that thou then proceed to three. Five is right out. Once the number three, being the third number, be reached, then lobbest thou thy Holy Failed Disk of NetApp towards thy foe, who, being naughty in my sight, shall snuff it.'
Until next time...
The Mathworks, Inc. 508-647-7000 x7792 3 Apple Hill Drive, Natick, MA 01760-2098 508-647-7001 FAX tmerrill@mathworks.com http://www.mathworks.com ---
On Tue, 10 Oct 2000, Todd C. Merrill wrote:
My questions then are:
Darn. Forgot one:
o What happens with the consistency points where there is more than one *volume* on the filer?
Until next time...
The Mathworks, Inc. 508-647-7000 x7792 3 Apple Hill Drive, Natick, MA 01760-2098 508-647-7001 FAX tmerrill@mathworks.com http://www.mathworks.com ---
"WAFL writes data in a round-robin fashion to the RAID groups in a volume. Data is written to one RAID group at a time, one per consistency point.
It looks like there are notes in the course you took that need updating. Sorry about that.
This was true in the earlier MV/MR capable ONTAP microkernels circa 5.0, but this isn't how things have worked for some time (around two years would be my guess). The above methodology created performance issues when one RAID group in the system was of dimunitive proportions. Things now work in a much more balanced manner across all the RAID groups in a volume, virtually eliminating the need for customers to be overly concerned with or even aware of such matters, which is of course how it should be.
o Is it even a good idea to have more than one RAID group per volume?
The rule of thumb is that the filer's defaults rock. Unless there is something particularly exceptional or peculiar in your environment, I'd leave them alone and just let Data ONTAP do its job. It's an expert! :-)
Say I fill up a volume on a 14-disk RAID group (12d+1p+1hs), and then I add two more shelves (13d+1p) in another RAID group to expand this volume. (This gets rid of the RAID group size imbalance that will definitely result in poorer performance, which is the next subject discussed after the quote above.) Say I continue to add data to the volume, and as is likely, the last added data is probably the most often used. Will I have a "hot spot" in the second RAID group with the consistency points, since I'll be doing lots of reads and writes to the new data and very little to the old data?
Probably for a while, but it's not really as simple as this. As time goes by, WAFL's copy-on-write habits would tend to "move" data that is in the first RAID group out into the second, at least for a while. For example, the metadata associated with the "old data", inodes that contain changing last access times, metadata files in general and the pointer blocks that track them and so forth, would tend to be hauled out into more spacious surrounds as WAFL's tree-'o-blocks structure changes. Even such things as snapshots cycling out of the system would likely create some decent expanses of room in that first RAID group not long after the second was added. There's all kind of things to consider, but in the vast majority of scenarios you should ultimately wind up with a very well automagically balanced state of affairs.
I believe your other questions were made moot by my telling you that the old round-robin algorithms are a thing of the distant past?
Keith
On Tue, 10 Oct 2000, Keith Brown wrote:
I believe your other questions were made moot by my telling you that the old round-robin algorithms are a thing of the distant past?
Indeed. "Pay no attention to that man behind the curtain."
Until next time...
The Mathworks, Inc. 508-647-7000 x7792 3 Apple Hill Drive, Natick, MA 01760-2098 508-647-7001 FAX tmerrill@mathworks.com http://www.mathworks.com ---
-
Keith Brown -
Rick Hulsey -
Todd C. Merrill