I was told by a Netapp rep that you can not mix 18Gb and 36Gb drives on a F720. This is not true is it as long as you use the correct disk shelves is it? Are there certain memory requirments for the 36Gb drives?
Thanks
Jason Middlebrooks Systems Engineer Datalink Corp. 888-933-9327 x2970
Just don't use the mix of 18 and 36Gb drive in the SAME volume. That is bad. I don't see why you can't have different disk shelves of same drives, ie. a shelf of 18s and another shelf of 36s, attached to an F720. We have this config on an F760.
-Scott
On Fri, 6 Oct 2000 jmiddlebrooks@datalink.com wrote:
I was told by a Netapp rep that you can not mix 18Gb and 36Gb drives on a F720. This is not true is it as long as you use the correct disk shelves is it? Are there certain memory requirments for the 36Gb drives?
Thanks
Jason Middlebrooks Systems Engineer Datalink Corp. 888-933-9327 x2970
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Scott T. Mikusko The stars are laughing at us, as we Systems Engineer crawl on and on across this antheap.... Concentric Network Corporation Internet Guerilla -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Just don't use the mix of 18 and 36Gb drive in the SAME volume. That is bad. I don't see why you can't have different disk shelves of same drives, ie. a shelf of 18s and another shelf of 36s, attached to an F720. We have this config on an F760.
NO, what you must not do is mix disk sizes in a *raid group*. Its fine to mix in a volume as long as they're in different raidgroups. I think the confusion arises as many peoples volumes don't span raidgroups. Another thing I would advise against is mixing sizes in a shelf.
chris_good@webtop.com writes:
Just don't use the mix of 18 and 36Gb drive in the SAME volume. That is bad. I don't see why you can't have different disk shelves of same drives, ie. a shelf of 18s and another shelf of 36s, attached to an F720. We have this config on an F760.
NO, what you must not do is mix disk sizes in a *raid group*. Its fine to mix in a volume as long as they're in different raidgroups. I think the confusion arises as many peoples volumes don't span raidgroups. Another thing I would advise against is mixing sizes in a shelf.
Seperating different disc sizes into different RAID groups avoids various sorts of disc space wastage in the short term. But it doesn't alter the fact that if you have a mixture of disc sizes within a volume, you are stuck with that state for as long as the volume exists: you can never expand a data plane to effectively use a larger disc.
Chris Thompson University of Cambridge Computing Service, Email: cet1@ucs.cam.ac.uk New Museums Site, Cambridge CB2 3QG, Phone: +44 1223 334715 United Kingdom.
----- Original Message ----- From: "Chris Thompson" cet1@cus.cam.ac.uk To: chris_good@webtop.com Cc: toasters@mathworks.com Sent: Friday, October 06, 2000 4:54 PM Subject: Re: mixing 18GB and 36GB drives
chris_good@webtop.com writes:
Just don't use the mix of 18 and 36Gb drive in the SAME volume. That is bad. I don't see why you can't have different disk shelves of same drives, ie. a shelf of 18s and another shelf of 36s, attached to an
F720.
We have this config on an F760.
NO, what you must not do is mix disk sizes in a *raid group*. Its fine to mix in a volume as long as they're in different raidgroups. I think the confusion arises as many peoples volumes don't span
raidgroups.
Another thing I would advise against is mixing sizes in a shelf.
Seperating different disc sizes into different RAID groups avoids various sorts of disc space wastage in the short term. But it doesn't alter the fact that if you have a mixture of disc sizes within a volume, you are stuck with that state for as long as the volume exists: you can never expand a data plane to effectively use a larger disc.
What are you people talking about?
If you have a raid group of 18GB drives, and add a 36GB drive, the 36GB drive becomes the new parity drive. The old 18GB parity drive becomes a data drive, giving you 18GB of space. After that, all 36GB drives you add give you the full 36GB of space.
(Caveat: This is how Netapp's RAID 4 works. With other vendors, YMMV.)
Bruce
If you have a raid group of 18GB drives, and add a 36GB drive, the 36GB drive becomes the new parity drive. The old 18GB parity drive becomes a data drive, giving you 18GB of space. After that, all 36GB drives you add give you the full 36GB of space.
The "problem" with this is that you end up with hot spots - with (for example) 4x18GB drives and 3x36GB drives in a single raid group, you'd end up with some parts of the volume where the stripe width is narrower, potentially limiting performance. The "lower half" of the filesystem would be striped across all 6 data drives (the 4 18's and 2 36's) but writes to the "upper half" would only touch the 2 36GB data drives.
FWIW, we (hi, Darrell!) helped discover this about 5 years ago on an F330 that was grown from 2GB drives to a mix of 2GB and 4GB drives. (We beat the snot out of that box; it averaged over 700 NFS ops/sec 24x7 for over two years...) Most of the data on the filesystem before the addition of the larger capacity drives was fairly static, and that exascerbated the problem quite a bit, since all new writes were spread over fewer drives. Empirical studies (i.e., watching the blinky lights!) highlighted the problem quite clearly. Perhaps backing up and restoring the volume would help that a bit, but what a pain...
So that's my recollection of why Netapp recommends against mixing drive sizes within a raid group. It's possible, but it may have a performance impact.
I'm sure someone will leap in and correct me if I'm wrong. :-)
Ta,
-- Chris
-- Chris Lamb, Unix Guy MeasureCast, Inc. 503-241-1469 x247 skeezics@measurecast.com
----- Original Message ----- From: "Chris Lamb" skeezics@measurecast.com To: toasters@mathworks.com Sent: Sunday, October 08, 2000 10:38 AM Subject: Re: mixing 18GB and 36GB drives
If you have a raid group of 18GB drives, and add a 36GB drive, the 36GB drive becomes the new parity drive. The old 18GB parity drive becomes a data drive, giving you 18GB of space. After that, all 36GB drives you add give you the full 36GB of space.
The "problem" with this is that you end up with hot spots
I'm quite aware of the "hot spot" problem. The poster said there was a space issue:
"Seperating different disc sizes into different RAID groups avoids various sorts of disc space wastage in the short term. But it doesn't alter the fact that if you have a mixture of disc sizes within a volume, you are stuck with that state for as long as the volume exists: you can never expand a data plane to effectively use a larger disc."
There's no space wastage and no problem expanding the data plane.
Bruce
If you have a raid group of 18GB drives, and add a 36GB drive, the 36GB drive becomes the new parity drive. The old 18GB parity drive becomes a data drive, giving you 18GB of space. After that, all 36GB drives you add give you the full 36GB of space.
(Caveat: This is how Netapp's RAID 4 works. With other vendors, YMMV.)
Bruce
Beware that if an 18G drive fails and the only available hot spare is a 36G drive, then only 18G of the 36G spare drive is used and 18G is wasted. This situation is permanent until the volume is destroyed, at least under DOT 5.3.x. Fortunately, hot spares are not tied to any particular volume or raid group, so one 18G hot spare can replace 18G drives from multiple volumes or raid groups.
Steve Losen scl@virginia.edu phone: 804-924-0640
University of Virginia ITC Unix Support
----- Original Message ----- From: "Steve Losen" scl@sasha.acc.virginia.edu To: toasters@mathworks.com Sent: Monday, October 09, 2000 5:55 AM Subject: Re: mixing 18GB and 36GB drives
If you have a raid group of 18GB drives, and add a 36GB drive, the 36GB drive becomes the new parity drive. The old 18GB parity drive becomes a data drive, giving you 18GB of space. After that, all 36GB drives you add give you the full 36GB of space.
(Caveat: This is how Netapp's RAID 4 works. With other vendors, YMMV.)
Bruce
Beware that if an 18G drive fails and the only available hot spare is a 36G drive, then only 18G of the 36G spare drive is used and 18G is wasted. This situation is permanent until the volume is destroyed, at least under DOT 5.3.x.
I thought you could, if you wanted to, get a new 18G drive, fail the 36G drive, and replace it with an 18G drive and the software was smart enough to reconstruct. But, I could be wrong, and of course you put yourself at risk and reduced performance during the reconstruction.
Bruce
On Mon, 9 Oct 2000, Bruce Sterling Woodcock wrote:
----- Original Message ----- From: "Steve Losen" scl@sasha.acc.virginia.edu
If you have a raid group of 18GB drives, and add a 36GB drive, the 36GB drive becomes the new parity drive. The old 18GB parity drive becomes a data drive, giving you 18GB of space. After that, all 36GB drives you add give you the full 36GB of space.
(Caveat: This is how Netapp's RAID 4 works. With other vendors, YMMV.)
Bruce
Beware that if an 18G drive fails and the only available hot spare is a 36G drive, then only 18G of the 36G spare drive is used and 18G is wasted. This situation is permanent until the volume is destroyed, at least under DOT 5.3.x.
I thought you could, if you wanted to, get a new 18G drive, fail the 36G drive, and replace it with an 18G drive and the software was smart enough to reconstruct. But, I could be wrong, and of course you put yourself at risk and reduced performance during the reconstruction.
Ooooo...crap. Could someone from NetApp please officially comment on the expected behaviour?
I've got a filer with a shelf of 36's (one RG/volume) and two shelves of 18's (one RG/volume); each RG has a hot spare. What I do NOT want to happen is the situation described in this thread--an 18 GB disk dying, ONTAP taking the 36 GB drive in as the new parity drive, and my not being able to fail it out because it is now a *full* 36 GB parity drive, instead of an 18 GB parity and 18 GB wasted.
An answer of "upgrade your old drives" is not acceptable. ;)
Until next time...
The Mathworks, Inc. 508-647-7000 x7792 3 Apple Hill Drive, Natick, MA 01760-2098 508-647-7001 FAX tmerrill@mathworks.com http://www.mathworks.com ---
At 11:24 AM 10/9/00 -0400, Todd C. Merrill wrote:
I've got a filer with a shelf of 36's (one RG/volume) and twoshelves of 18's (one RG/volume); each RG has a hot spare. What I do NOT want to happen is the situation described in this thread--an 18 GB disk dying, ONTAP taking the 36 GB drive in as the new parity drive, and my not being able to fail it out because it is now a *full* 36 GB parity drive, instead of an 18 GB parity and 18 GB wasted.
If you have an 18GB drive fail and there are only 36GB hot spares, the filer will automatically take a 36GB drive and begin rebuilding, but it will only use 18GB of the available space. If this bothers you, you could later add a spare 18GB drive to the system, pull the 36GB drive that's in your 18GB volume, and the filer will rebuild using the new 18GB spare. If you don't like the risk of forcing a rebuild in this manner, it's best to keep hot spares for each size disk in the filer.
This situation is different from purposely expanding an 18GB volume/RG (vol add) with 36GB disks.
Hope that helps.
-- James N. (Jamey) Maze Office: 615-773-3541 System Engineer Mobile: 615-496-4799 Network Appliance www.netapp.com
On Mon, 9 Oct 2000, Jamey Maze wrote:
At 11:24 AM 10/9/00 -0400, Todd C. Merrill wrote:
I've got a filer with a shelf of 36's (one RG/volume) and twoshelves of 18's (one RG/volume); each RG has a hot spare. What I do
^^^^^^^^^^^^^^^^^^^^^^^
If you have an 18GB drive fail and there are only 36GB hot spares, the filer will automatically take a 36GB drive and begin rebuilding, but it
What happens when there is both a 36 GB and an 18 GB hot spare available? Is is a crapshoot?
Until next time...
The Mathworks, Inc. 508-647-7000 x7792 3 Apple Hill Drive, Natick, MA 01760-2098 508-647-7001 FAX tmerrill@mathworks.com http://www.mathworks.com ---
At 12:47 PM 10/9/00 -0400, Todd C. Merrill wrote:
At 11:24 AM 10/9/00 -0400, Todd C. Merrill wrote:
I've got a filer with a shelf of 36's (one RG/volume) and twoshelves of 18's (one RG/volume); each RG has a hot spare. What I do
^^^^^^^^^^^^^^^^^^^^^^^If you have an 18GB drive fail and there are only 36GB hot spares, the filer will automatically take a 36GB drive and begin rebuilding, but it
What happens when there is both a 36 GB and an 18 GB hot spareavailable? Is is a crapshoot?
My bad. If an 18GB drive fails and there's an 18GB hot spare, it'll always use it.
-- James N. (Jamey) Maze Office: 615-773-3541 System Engineer Mobile: 615-496-4799 Network Appliance www.netapp.com
Chris Lamb (skeezics@measurecast.com) wrote:
FWIW, we (hi, Darrell!) helped discover this about 5 years ago on an F330 that was grown from 2GB drives to a mix of 2GB and 4GB drives. (We beat the snot out of that box; it averaged over 700 NFS ops/sec 24x7 for over two years...)
At the risk of starting a "mine-is-bigger-than-yours" or a "I've-got-it-worse-than-you" [0] war...
That's not beating the snot out of an F330. *This* is (see Apr/May timeframe):
ftp://ftp.mathworks.com/pub/tech-support/incoming/toddc/F330.nfs.snmp-year.gif
MRTG one-year graph, one-day average: MaxNFS Ops 1713.0 ops, AverageNFS Ops 1047.0 ops, CurrentNFS Ops 717.0 ops
Add another ~200-250 ops/s for CIFS traffic, too.
We hope the replacement F840 arrives before this poor F330 bursts into flames. With that kind of traffic, we're gonna have to find more stuff to put on that filer 'cause the F840 is going to be bored out of its mind...
(We stopped the NFS traffic growth by implementing CacheFS mounts on many of our UNIX boxes. <whew!> That bought us quite a bit of time.)
[0] Tried to find a link to the Monty Python sketch with the old guys discussing how bad they had it during their childhoods, but came up empty.
Until next time...
The Mathworks, Inc. 508-647-7000 x7792 3 Apple Hill Drive, Natick, MA 01760-2098 508-647-7001 FAX tmerrill@mathworks.com http://www.mathworks.com ---
[0] Tried to find a link to the Monty Python sketch with the old guys discussing how bad they had it during their childhoods, but came up empty.
NetApp delivers:
Script -- http://www.montypython.net/hbowlmm.php3#4 Men Recording -- http://www.montypython.net/cgi-bin/dl/full.cgi?4york.wav
That's not beating the snot out of an F330. *This* is (see Apr/May timeframe):
MaxNFS Ops 1713.0 ops, AverageNFS Ops 1047.0 ops, CurrentNFS Ops 717.0 ops
Heck, that's nothing. A few years ago, I had a 330 doing >4000 nfs ops per second. Our salesman and field engineer were witnesses - they took a snapshot of the sysstat output and passed it around the office, since no one had ever seen a 330 stand up to that kind of load before.
I also had a 760 survive doing 17,000 nfs ops/second once. Our engineers are *really* good at beating up file servers - luckily, I got really good at beating up our engineers and our load levels are a bit more balanced now.
Hal Siegel _---_ / Senior Systems Engineer(ing) YY()))))\ /| Texas Microprocessor Division (@@) )))))/ Advanced Micro Devices /_/__/__/ hal@beast.amd.com mm mm
If an 18 gig drive fails, and the filer has an 18 gig spare, it will use the 18, otherwise it will rebuild on the 36 gig drive.
Jack ----- Original Message ----- From: "Todd C. Merrill" tmerrill@mathworks.com To: toasters@mathworks.com Sent: Monday, October 09, 2000 8:24 AM Subject: Re: mixing 18GB and 36GB drives
On Mon, 9 Oct 2000, Bruce Sterling Woodcock wrote:
----- Original Message ----- From: "Steve Losen" scl@sasha.acc.virginia.edu
If you have a raid group of 18GB drives, and add a 36GB drive, the 36GB drive becomes the new parity drive. The old 18GB parity drive becomes a data drive, giving you 18GB of space. After that, all
36GB
drives you add give you the full 36GB of space.
(Caveat: This is how Netapp's RAID 4 works. With other vendors,
YMMV.)
Bruce
Beware that if an 18G drive fails and the only available hot spare is a 36G drive, then only 18G of the 36G spare drive is used and 18G is wasted. This situation is permanent until the volume is destroyed, at least under DOT 5.3.x.
I thought you could, if you wanted to, get a new 18G drive, fail the 36G drive, and replace it with an 18G drive and the software was smart enough to reconstruct. But, I could be wrong, and of course you put yourself at risk and reduced performance during the reconstruction.
Ooooo...crap. Could someone from NetApp please officially comment on the expected behaviour?
I've got a filer with a shelf of 36's (one RG/volume) and two shelves of 18's (one RG/volume); each RG has a hot spare. What I do NOT want to happen is the situation described in this thread--an 18 GB disk dying, ONTAP taking the 36 GB drive in as the new parity drive, and my not being able to fail it out because it is now a *full* 36 GB parity drive, instead of an 18 GB parity and 18 GB wasted.
An answer of "upgrade your old drives" is not acceptable. ;)
Until next time...
The Mathworks, Inc. 508-647-7000 x7792 3 Apple Hill Drive, Natick, MA 01760-2098 508-647-7001 FAX tmerrill@mathworks.com http://www.mathworks.com
Hi, You are right in that the 36 would replace the 18 parity disk and 18 GB would be lost. However, the situation is NOT permanent as you could insert a new 18GB spare disk, fail the 36 parity disk and have it rebuild on the new 18 hot spare.
Regards, Eli ----- Original Message ----- From: Steve Losen scl@sasha.acc.virginia.edu To: toasters@mathworks.com Sent: Monday, October 09, 2000 2:55 PM Subject: Re: mixing 18GB and 36GB drives
If you have a raid group of 18GB drives, and add a 36GB drive, the 36GB drive becomes the new parity drive. The old 18GB parity drive becomes a data drive, giving you 18GB of space. After that, all 36GB drives you add give you the full 36GB of space.
(Caveat: This is how Netapp's RAID 4 works. With other vendors, YMMV.)
Bruce
Beware that if an 18G drive fails and the only available hot spare is a 36G drive, then only 18G of the 36G spare drive is used and 18G is wasted. This situation is permanent until the volume is destroyed, at least under DOT 5.3.x. Fortunately, hot spares are not tied to any particular volume or raid group, so one 18G hot spare can replace 18G drives from multiple volumes or raid groups.
Steve Losen scl@virginia.edu phone: 804-924-0640
University of Virginia ITC Unix Support
-
Bruce Sterling Woodcock -
Chris Lamb -
Chris Thompson -
chris_good＠webtop.com -
Eli Lopez -
Hal Siegel -
Jack Verdirame -
Jamey Maze -
jmiddlebrooks＠datalink.com -
Michael J. Tuciarone -
Scott Mikusko -
Steve Losen -
Todd C. Merrill