I'm already spreading disks in a volume across the 7 SCSI controllers, but I'm wondering if there is also a performance advantage/disadvantage to spreading out the disks in a volume across the three IDE-to-SCSI bridges in each R100 disk shelf?
For example, if I wanted to create a volume with 3 disks, would it be better to pick three disks on the same bridge (like 3.0, 3.1, and 3.2) or would it be better to pick three across three bridges (like 3.0, 3.8, and 3.12) ? Or doesn't it make a difference?
Thanks,
Mark
Mark Bentley wrote:
I'm already spreading disks in a volume across the 7 SCSI controllers, but I'm wondering if there is also a performance advantage/disadvantage to spreading out the disks in a volume across the three IDE-to-SCSI bridges in each R100 disk shelf?
yes, per my testing. The scsi-to-ATA bridegs appear to serialize transfers. Also, ATA bridges can fail, so IMHO you want to arrange your raid-groups so that any ATA brige failure can only take out one spindle in a raid-group, and the minimum number of spindles in any given volume.
What I ended up doing is somthing like this. vol2 is all the .14, .10 and .2 spindles, vol1 is all the .13, .9 and .1 spindles. vol 4 is just the .12 and .8 spindles (smaller), vol0 (root volume) is 3 of the .0 spindles, and four of the .0 spindles are spares.
the bitch is that, when a drive fails and rebuilds to a spare, you have to re-rebuild the volume to get the raid-groups back into alingment.
Works for me, but YMMV.
Volume vol1 (online, normal) (block checksums) Plex /vol1/plex0 (online, normal, active) RAID group /vol1/plex0/rg0 (normal)
RAID Disk Device HA SHELF BAY CHAN Used (MB/blks) Phys (MB/blks) --------- ------------ --------------- ---- -------------- -------------- parity 9.15 9 0 11 136000/278528000 138959/284589376 data 7.15 7 0 11 136000/278528000 138959/284589376 data 6.15 6 0 11 136000/278528000 138959/284589376 data 10.15 10 0 11 136000/278528000 138959/284589376 data 8.15 8 0 11 136000/278528000 138959/284589376 data 3.15 3 0 11 136000/278528000 138959/284589376 data 2.15 2 0 11 136000/278528000 138959/284589376
RAID group /vol1/plex0/rg1 (normal)
RAID Disk Device HA SHELF BAY CHAN Used (MB/blks) Phys (MB/blks) --------- ------------ --------------- ---- -------------- -------------- parity 7.11 7 0 7 136000/278528000 138959/284589376 data 9.11 9 0 7 136000/278528000 138959/284589376 data 8.11 8 0 7 136000/278528000 138959/284589376 data 10.11 10 0 7 136000/278528000 138959/284589376 data 6.11 6 0 7 136000/278528000 138959/284589376 data 3.11 3 0 7 136000/278528000 138959/284589376 data 2.11 2 0 7 136000/278528000 138959/284589376
RAID group /vol1/plex0/rg2 (normal)
RAID Disk Device HA SHELF BAY CHAN Used (MB/blks) Phys (MB/blks) --------- ------------ --------------- ---- -------------- -------------- parity 7.3 7 0 3 136000/278528000 138959/284589376 data 9.3 9 0 3 136000/278528000 138959/284589376 data 8.3 8 0 3 136000/278528000 138959/284589376 data 10.3 10 0 3 136000/278528000 138959/284589376 data 6.3 6 0 3 136000/278528000 138959/284589376 data 3.3 3 0 3 136000/278528000 138959/284589376 data 2.3 2 0 3 136000/278528000 138959/284589376
(e.g.
For example, if I wanted to create a volume with 3 disks, would it be better to pick three disks on the same bridge (like 3.0, 3.1, and 3.2) or would it be better to pick three across three bridges (like 3.0, 3.8, and 3.12) ? Or doesn't it make a difference?
Thanks,
Mark
On Thu, 27 Feb 2003, Skottie Miller wrote:
What I ended up doing is somthing like this. vol2 is all the .14, .10 and .2 spindles, vol1 is all the .13, .9 and .1 spindles. vol 4 is just the .12 and .8 spindles (smaller), vol0 (root volume) is 3 of the .0 spindles, and four of the .0 spindles are spares.
Thanks for the responce Skottie, one more question: how did you configure vol0 to use 3 of the .0 spindles? By default, vol0 is created on 2.0 and 2.1. I'd like to have vol0 on, say, 2.0 and 3.0.
Thanks,
Mark
Mark Bentley wrote:
On Thu, 27 Feb 2003, Skottie Miller wrote:
What I ended up doing is somthing like this. vol2 is all the .14, .10 and .2 spindles, vol1 is all the .13, .9 and .1 spindles. vol 4 is just the .12 and .8 spindles (smaller), vol0 (root volume) is 3 of the .0 spindles, and four of the .0 spindles are spares.
Thanks for the responce Skottie, one more question: how did you configure vol0 to use 3 of the .0 spindles? By default, vol0 is created on 2.0 and 2.1. I'd like to have vol0 on, say, 2.0 and 3.0.
I moved it. If you can handle a bit of downtime, you can create a new vol0, copy stuff, change it to the new root volume, reboot. or physicially move the spindle; drives don't know what slot they are in. If you can't handle downtime, you can "move" the volume by doing somthing like this: - make sure that the only spare disk is 3.0 - disk fail 2.1, forcing it to rebuild onto 3.0 - after it rebuilds, disk unfail 2.1
you might want to turn of autosupport as you're failing disks, otherwise you'll open extra support calls.
-skottie
-
Mark Bentley -
Skottie Miller