I'm the proud new owner of an IBM N3600 A20 (rebranded FAS2050C) with 20x30GB SAS disks.
I'm trying to determine the best way to get this thing set up, and realized I have only a bit of a fuzzy understanding as to how the clustering or failover filer head should work.
My initial thoughts were to aim for the following setup:
- Set up all 20 disks in a RAID-DP aggregate with one spare (17 data, 2 parity and one spare, or maybe two spares). - Bond a NIC from the first controller with a NIC from the second controller to give us a 2Gbps connection to our "storage network". - Third and fourth NIC's would go to our regular network.
My hope was that I could lose one filer head and the other would take over seamlessly. We'd lose half of our network bandwidth but still be up and running.
However, it sounds like my understanding of how the clustering works might have been a bit flawed and that I actually need to treat the filer heads as two separate filers. So I may be forced to do something like the following:
- Split my disks up between the two filers (7 data, 2 parity, one spare -- or maybe I can have one spare available to both heads). - Probably can't team NIC's from multiple filer heads meaning if I team the two NIC's on the filer I can no longer connect to my management network. I probably need to order more NIC's :( - If I lose one head, I lose one aggregate unless manual intervention is taken. - Each filer has a different hostname/IP for network access.
This maybe gives me better performance, but at the expense of total disk space and flexibility if my understanding is correct.
Maybe someone could help clear this up. It doesn't appear IBM has a RedBook on clustering... I'm searching around in NOW and have come across the Data ONTAP 7.3 Active/Active Configuration Guide which I am now reading.
Is there something similar for Active/Passive setups (which seems to be more what I am after) or other documents that would be recommended reading? Any advice or best practices?
This filer will be serving NFS to a pair of ESX servers. We plan to add a second shelf of disks later this year.
Thanks in advance. No sales inquiries please.
Ray
You are correct that clustering is treated as two separate controllers which can take over for each other. You cannot vif across NICs on different controllers.
If you want to do the closest thing to active/passive would be to allocate at least 2 (possible 3 if you want a spare) disks to the "passive" controller and the rest to the active one. I'd set up a raid4 trad vol or aggregate for it since you only are going to use 2 disks, you don't need raid_dp. Definitely use raid_dp on the active controller.
Under this scenario, you can lose either controller head and still be running.
-- Adam Fox Systems Engineer adamfox@netapp.com
-----Original Message----- From: Ray Van Dolson [mailto:rvandolson@esri.com] Sent: Tuesday, June 02, 2009 11:44 AM To: toasters@mathworks.com Subject: FAS2050C questions (clustering)
I'm the proud new owner of an IBM N3600 A20 (rebranded FAS2050C) with 20x30GB SAS disks.
I'm trying to determine the best way to get this thing set up, and realized I have only a bit of a fuzzy understanding as to how the clustering or failover filer head should work.
My initial thoughts were to aim for the following setup:
- Set up all 20 disks in a RAID-DP aggregate with one spare (17 data, 2 parity and one spare, or maybe two spares). - Bond a NIC from the first controller with a NIC from the second controller to give us a 2Gbps connection to our "storage network". - Third and fourth NIC's would go to our regular network.
My hope was that I could lose one filer head and the other would take over seamlessly. We'd lose half of our network bandwidth but still be up and running.
However, it sounds like my understanding of how the clustering works might have been a bit flawed and that I actually need to treat the filer heads as two separate filers. So I may be forced to do something like the following:
- Split my disks up between the two filers (7 data, 2 parity, one spare -- or maybe I can have one spare available to both heads). - Probably can't team NIC's from multiple filer heads meaning if I team the two NIC's on the filer I can no longer connect to my management network. I probably need to order more NIC's :( - If I lose one head, I lose one aggregate unless manual intervention is taken. - Each filer has a different hostname/IP for network access.
This maybe gives me better performance, but at the expense of total disk space and flexibility if my understanding is correct.
Maybe someone could help clear this up. It doesn't appear IBM has a RedBook on clustering... I'm searching around in NOW and have come across the Data ONTAP 7.3 Active/Active Configuration Guide which I am now reading.
Is there something similar for Active/Passive setups (which seems to be more what I am after) or other documents that would be recommended reading? Any advice or best practices?
This filer will be serving NFS to a pair of ESX servers. We plan to add a second shelf of disks later this year.
Thanks in advance. No sales inquiries please.
Ray
On Tue, Jun 02, 2009 at 08:54:35AM -0700, Fox, Adam wrote:
You are correct that clustering is treated as two separate controllers which can take over for each other. You cannot vif across NICs on different controllers.
If you want to do the closest thing to active/passive would be to allocate at least 2 (possible 3 if you want a spare) disks to the "passive" controller and the rest to the active one. I'd set up a raid4 trad vol or aggregate for it since you only are going to use 2 disks, you don't need raid_dp. Definitely use raid_dp on the active controller.
Under this scenario, you can lose either controller head and still be running.
Ah, so we need to have disks assigned to the "passive" controller in an aggregate configuration? What if I just split the disks up evenly, would the aggregate on "active" controller shift down to be controlled by the "passive" controller automatically?
Maybe this would be preferrable to having 2 or 3 disks doing "nothing" on the second head.
Thanks for the response.
-- Adam Fox Systems Engineer adamfox@netapp.com
Ray
You can split them if you like. I only said do the 2-disk to the "passive" side if you wanted an active/passive config. If you want to go active/active, then split them up. Just be aware that depending on your load, you may get spindle-bound at some point with that few disks in your aggregate, but you may be fine until you get your new disks later.
-- Adam Fox Systems Engineer adamfox@netapp.com
-----Original Message----- From: Ray Van Dolson [mailto:rvandolson@esri.com] Sent: Tuesday, June 02, 2009 12:01 PM To: Fox, Adam Cc: toasters@mathworks.com Subject: Re: FAS2050C questions (clustering)
On Tue, Jun 02, 2009 at 08:54:35AM -0700, Fox, Adam wrote:
You are correct that clustering is treated as two separate controllers which can take over for each other. You cannot vif across NICs on different controllers.
If you want to do the closest thing to active/passive would be to allocate at least 2 (possible 3 if you want a spare) disks to the "passive" controller and the rest to the active one. I'd set up a
raid4
trad vol or aggregate for it since you only are going to use 2 disks, you don't need raid_dp. Definitely use raid_dp on the active controller.
Under this scenario, you can lose either controller head and still be running.
Ah, so we need to have disks assigned to the "passive" controller in an aggregate configuration? What if I just split the disks up evenly, would the aggregate on "active" controller shift down to be controlled by the "passive" controller automatically?
Maybe this would be preferrable to having 2 or 3 disks doing "nothing" on the second head.
Thanks for the response.
-- Adam Fox Systems Engineer adamfox@netapp.com
Ray
Depending on your workload you could get a decent boost by splitting it up since that means you'll have twice as much cache servicing your requests. Keep in mind it's not just about the disks :)
-----Original Message----- From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Fox, Adam Sent: Tuesday, June 02, 2009 12:03 PM To: Ray Van Dolson Cc: toasters@mathworks.com Subject: RE: FAS2050C questions (clustering)
You can split them if you like. I only said do the 2-disk to the "passive" side if you wanted an active/passive config. If you want to go active/active, then split them up. Just be aware that depending on your load, you may get spindle-bound at some point with that few disks in your aggregate, but you may be fine until you get your new disks later.
-- Adam Fox Systems Engineer adamfox@netapp.com
-----Original Message----- From: Ray Van Dolson [mailto:rvandolson@esri.com] Sent: Tuesday, June 02, 2009 12:01 PM To: Fox, Adam Cc: toasters@mathworks.com Subject: Re: FAS2050C questions (clustering)
On Tue, Jun 02, 2009 at 08:54:35AM -0700, Fox, Adam wrote:
You are correct that clustering is treated as two separate controllers which can take over for each other. You cannot vif across NICs on different controllers.
If you want to do the closest thing to active/passive would be to allocate at least 2 (possible 3 if you want a spare) disks to the "passive" controller and the rest to the active one. I'd set up a
raid4
trad vol or aggregate for it since you only are going to use 2 disks, you don't need raid_dp. Definitely use raid_dp on the active controller.
Under this scenario, you can lose either controller head and still be running.
Ah, so we need to have disks assigned to the "passive" controller in an aggregate configuration? What if I just split the disks up evenly, would the aggregate on "active" controller shift down to be controlled by the "passive" controller automatically?
Maybe this would be preferrable to having 2 or 3 disks doing "nothing" on the second head.
Thanks for the response.
-- Adam Fox Systems Engineer adamfox@netapp.com
Ray
Please be advised that this email may contain confidential information. If you are not the intended recipient, please do not read, copy or re-transmit this email. If you have received this email in error, please notify us by email by replying to the sender and by telephone (call us collect at +1 202-828-0850) and delete this message and any attachments. Thank you in advance for your cooperation and assistance.
In addition, Danaher and its subsidiaries disclaim that the content of this email constitutes an offer to enter into, or the acceptance of, any contract or agreement or any amendment thereto; provided that the foregoing disclaimer does not invalidate the binding effect of any digital or other electronic reproduction of a manual signature that is included in any attachment to this email.
You need more NICs (we hit this issue all the time).
Basically, set both filers up on both networks as if they were separate filers. Given them separate IP addresses. If one head dies, the other head will takeover all connections and assume the "personality" of the dead filers. So the IPs and the disks of the dead filer will all be visile on the live one.
The heads have an internal interconenct that makes this possible.
And everything Adam said about the disks.
Peta
2009/6/2 Fox, Adam Adam.Fox@netapp.com
You are correct that clustering is treated as two separate controllers which can take over for each other. You cannot vif across NICs on different controllers.
If you want to do the closest thing to active/passive would be to allocate at least 2 (possible 3 if you want a spare) disks to the "passive" controller and the rest to the active one. I'd set up a raid4 trad vol or aggregate for it since you only are going to use 2 disks, you don't need raid_dp. Definitely use raid_dp on the active controller.
Under this scenario, you can lose either controller head and still be running.
-- Adam Fox Systems Engineer adamfox@netapp.com
-----Original Message----- From: Ray Van Dolson [mailto:rvandolson@esri.com] Sent: Tuesday, June 02, 2009 11:44 AM To: toasters@mathworks.com Subject: FAS2050C questions (clustering)
I'm the proud new owner of an IBM N3600 A20 (rebranded FAS2050C) with 20x30GB SAS disks.
I'm trying to determine the best way to get this thing set up, and realized I have only a bit of a fuzzy understanding as to how the clustering or failover filer head should work.
My initial thoughts were to aim for the following setup:
- Set up all 20 disks in a RAID-DP aggregate with one spare (17 data, 2 parity and one spare, or maybe two spares).
- Bond a NIC from the first controller with a NIC from the second controller to give us a 2Gbps connection to our "storage network".
- Third and fourth NIC's would go to our regular network.
My hope was that I could lose one filer head and the other would take over seamlessly. We'd lose half of our network bandwidth but still be up and running.
However, it sounds like my understanding of how the clustering works might have been a bit flawed and that I actually need to treat the filer heads as two separate filers. So I may be forced to do something like the following:
- Split my disks up between the two filers (7 data, 2 parity, one spare -- or maybe I can have one spare available to both heads).
- Probably can't team NIC's from multiple filer heads meaning if I team the two NIC's on the filer I can no longer connect to my management network. I probably need to order more NIC's :(
- If I lose one head, I lose one aggregate unless manual intervention is taken.
- Each filer has a different hostname/IP for network access.
This maybe gives me better performance, but at the expense of total disk space and flexibility if my understanding is correct.
Maybe someone could help clear this up. It doesn't appear IBM has a RedBook on clustering... I'm searching around in NOW and have come across the Data ONTAP 7.3 Active/Active Configuration Guide which I am now reading.
Is there something similar for Active/Passive setups (which seems to be more what I am after) or other documents that would be recommended reading? Any advice or best practices?
This filer will be serving NFS to a pair of ESX servers. We plan to add a second shelf of disks later this year.
Thanks in advance. No sales inquiries please.
Ray
On Tue, Jun 02, 2009 at 05:04:29PM +0100, Peta Spies wrote:
You need more NICs (we hit this issue all the time).
Not necessarily. If you now have 2 NICs on both filers, simply VIF those NICs, then configure a vlan trunk over that etherchannel, and create vlan interfaces. Make sure to create interfaces for all vlans on both filers, so they can failover to eachother.
This assumes that your management network is actually just a vlan on the same infrastructure. If it's separate hardware, you do need more NICs
Ray Van Dolson wrote:
Gotcha. I'll just have to think about the RAID-4 vs RAID-6 thing then. Extra 300GB of space might be nice...
Don't. The extra headache you get when your performance goes down the drain whenever you have a high priority RAID rebuild, or the chance of data loss because there's a second failure during rebuild, REALLY outweighs the cost of a single extra disk. RAID-DP uses low priority rebuilds which take longer, but don't have such a big impact on your production data (unless you get into double degraded mode - which would have meant data loss on RAID-4).
Also, RAID-DP raid groups can be larger than RAID-4 raid groups, so you can more easily extend your RAID-DP based volumes (or aggregates), and if you do, RAID-DP is just as efficient as RAID-4, just vastly more reliable. It does add a little more overhead, but if you want raw performance, just use RAID-0, and recover from disk crashes using "newfs" (we do that - but not on netapp hardware obviously - and you'll have to add redundancy at another level).
-
Fox, Adam -
Jan Pieter Cornet -
Page, Jeremy -
Peta Spies -
Ray Van Dolson