Yeah, keep looking back at the FAS8020 for some reason...only has the four ports.
Make sure you have the X6599A (the 10-GIG SFP) and not the X6596(FC SFP). For each controller: a0a should consist of e0e/e0g which should go to the same switch, if possible, different blades on the same switch and e0f/e0h which should go to the same switch, if possible, different blades on the same switch --> if you cannot split across blades, at least split across ASICs on the same blade.
--tmac
*Tim McCarthy* *Principal Consultant*
On Thu, Apr 9, 2015 at 8:24 AM, Clark, André M. Andre.Clark@redeight.com wrote:
Each 8040/60/80 controller has 4 10GbE ports (e0a-e0d) and 4 UTA2 ports (e0e-e0h). Ports e0a-e0d, in a cDOT configuration, are used for the private cluster network while ports e0e-e0h are used for data communications. No extra 10G card is needed at this time.
Regards,
André M. Clark Andre.Clark@redeight.com | Sr. Consulting Architect | 917.388.8236
*Tell me I will forget... Show me I may remember... Involve me I WILL UNDERSTAND!!! Start by doing what's necessary, then what's possible, and suddenly you are doing the impossible!!!*
*Red8*, An Insight Investments Company
www.redeight.com
*From:* toasters-bounces@teaparty.net [mailto: toasters-bounces@teaparty.net] *On Behalf Of *tmac *Sent:* Thursday, 9 April, 2015 06:22 *To:* Francis Kim *Cc:* toasters@teaparty.net *Subject:* Re: FAS8040 network configuration in CDOT
No that won't work.
If you are configuring cDOT, at least two ports per controller are reserved for the cluster network (e0a & e0c)
This leaves e0b and e0d available for normal network access
The recommended configuration is to use all four ports for the cluster network (e0a, e0b, e0c & e0d).
In this case, if you need/want 10gig network connections you would need an extra 10gig card
AS indicated earlier in this thread, from each controller:
a0a should be a multimode_lacp VIF and contain:
e0b -> switch 1
e0d -> switch 2
Then you can create your VLANs: a0a-123, a0a-456
Place your LIFs on the tagged VLANs
Create a failover group for node-01:a0a-123 and node-02:a0a-123
Create a failover group for node-01:a0a-456 and node-02:a0a-456
Assign the failover group to the appropriate LIF
PS: DO NOT FORGET to disable spanning tree on the switch ports.
--> spanning tree protocol should only be enabled on ports connected to other switches.
--> it is not needed on ports connected to en hosts
--> in a cluster failover event it could take an extra 45 seconds for a port to become available.(not good)
--> remember to use the right version of the command. if it is a trunk, you usually need that on the line
--tmac
*Tim McCarthy*
*Principal Consultant*
On Wed, Apr 8, 2015 at 11:36 PM, Francis Kim fkim@berkcom.com wrote:
You can also tie the four links into a single interface group and carve it up into multiple VLANs.
What ports are you going to use for the cluster interconnect? I assume you're ordering a switchless cluster of two nodes?
*Francis Kim *| Engineer
510-644-1599 x334 | fkim@berkcom.com
*BerkCom* | www.berkcom.com
NetApp | Cisco | Supermicro | Brocade | VMware
On Apr 8, 2015, at 3:51 PM, Sayla, Mustafa MSAYLA@mesirowfinancial.com wrote:
We are looking to buy new FAS8040 and configure it with CDOT. I am looking for network recommendation with Cisco 6509 VSS. 8040 has four 10G ports per controller and I want to use all four ports to create an interface group with two VLAN, using VLAN tagging. I want 2 ports to go to one VSS switch and 2 ports to go to other VSS switch so I have switch redundancy also. I am not sure if it is doable since I am going to two different switches. Any recommendation is appreciated.
Mustafa
*Visit us on the Web at mesirowfinancial.com http://mesirowfinancial.com/*
*This communication may contain privileged and/or confidential information. It is intended solely for the use of the addressee. If you are not the intended recipient, you are strictly prohibited from disclosing, copying, distributing or using any of this information. If you received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. Confidential, proprietary or time-sensitive communications should not be transmitted via the Internet, as there can be no assurance of actual or timely delivery, receipt and/or confidentiality. This is not an offer, or solicitation of any offer to buy or sell any security, investment or other product.*_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Thank you for your response. I verified we have the X6599A SFP. Just to clarify what I meant to ask is that can we have two ports go to one pair of Catalyst 6500 switch and 2 ports go to another pair of Catalyst 6500 switch and tie all four ports into one interface group so I have 40GB and also have switch redundancy. If not possible what is the best configuration I can have to have 40GB throughput.
Mustafa Sayla
From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of tmac Sent: Thursday, April 09, 2015 7:52 AM To: Clark, André M. Cc: toasters@teaparty.net Subject: Re: FAS8040 network configuration in CDOT
Yeah, keep looking back at the FAS8020 for some reason...only has the four ports.
Make sure you have the X6599A (the 10-GIG SFP) and not the X6596(FC SFP). For each controller: a0a should consist of e0e/e0g which should go to the same switch, if possible, different blades on the same switch and e0f/e0h which should go to the same switch, if possible, different blades on the same switch --> if you cannot split across blades, at least split across ASICs on the same blade.
--tmac
Tim McCarthy Principal Consultant
On Thu, Apr 9, 2015 at 8:24 AM, Clark, André M. <Andre.Clark@redeight.commailto:Andre.Clark@redeight.com> wrote: Each 8040/60/80 controller has 4 10GbE ports (e0a-e0d) and 4 UTA2 ports (e0e-e0h). Ports e0a-e0d, in a cDOT configuration, are used for the private cluster network while ports e0e-e0h are used for data communications. No extra 10G card is needed at this time.
Regards,
André M. Clarkmailto:Andre.Clark@redeight.com | Sr. Consulting Architect | 917.388.8236tel:917.388.8236 Tell me I will forget... Show me I may remember... Involve me I WILL UNDERSTAND!!! Start by doing what's necessary, then what's possible, and suddenly you are doing the impossible!!! Red8, An Insight Investments Company www.redeight.comhttp://www.redeight.com/
From: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net] On Behalf Of tmac Sent: Thursday, 9 April, 2015 06:22 To: Francis Kim Cc: toasters@teaparty.netmailto:toasters@teaparty.net Subject: Re: FAS8040 network configuration in CDOT
No that won't work. If you are configuring cDOT, at least two ports per controller are reserved for the cluster network (e0a & e0c) This leaves e0b and e0d available for normal network access
The recommended configuration is to use all four ports for the cluster network (e0a, e0b, e0c & e0d). In this case, if you need/want 10gig network connections you would need an extra 10gig card
AS indicated earlier in this thread, from each controller: a0a should be a multimode_lacp VIF and contain: e0b -> switch 1 e0d -> switch 2
Then you can create your VLANs: a0a-123, a0a-456
Place your LIFs on the tagged VLANs
Create a failover group for node-01:a0a-123 and node-02:a0a-123 Create a failover group for node-01:a0a-456 and node-02:a0a-456
Assign the failover group to the appropriate LIF
PS: DO NOT FORGET to disable spanning tree on the switch ports. --> spanning tree protocol should only be enabled on ports connected to other switches. --> it is not needed on ports connected to en hosts --> in a cluster failover event it could take an extra 45 seconds for a port to become available.(not good) --> remember to use the right version of the command. if it is a trunk, you usually need that on the line
--tmac
Tim McCarthy Principal Consultant
On Wed, Apr 8, 2015 at 11:36 PM, Francis Kim <fkim@berkcom.commailto:fkim@berkcom.com> wrote: You can also tie the four links into a single interface group and carve it up into multiple VLANs.
What ports are you going to use for the cluster interconnect? I assume you're ordering a switchless cluster of two nodes?
Francis Kim | Engineer 510-644-1599 x334tel:510-644-1599%20x334 | fkim@berkcom.commailto:fkim@berkcom.com
BerkCom | www.berkcom.comhttp://www.berkcom.com/ NetApp | Cisco | Supermicro | Brocade | VMware
On Apr 8, 2015, at 3:51 PM, Sayla, Mustafa <MSAYLA@mesirowfinancial.commailto:MSAYLA@mesirowfinancial.com> wrote:
We are looking to buy new FAS8040 and configure it with CDOT. I am looking for network recommendation with Cisco 6509 VSS. 8040 has four 10G ports per controller and I want to use all four ports to create an interface group with two VLAN, using VLAN tagging. I want 2 ports to go to one VSS switch and 2 ports to go to other VSS switch so I have switch redundancy also. I am not sure if it is doable since I am going to two different switches. Any recommendation is appreciated.
Mustafa Visit us on the Web at mesirowfinancial.comhttp://mesirowfinancial.com/ This communication may contain privileged and/or confidential information. It is intended solely for the use of the addressee. If you are not the intended recipient, you are strictly prohibited from disclosing, copying, distributing or using any of this information. If you received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. Confidential, proprietary or time-sensitive communications should not be transmitted via the Internet, as there can be no assurance of actual or timely delivery, receipt and/or confidentiality. This is not an offer, or solicitation of any offer to buy or sell any security, investment or other product._______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Visit us on the Web at mesirowfinancial.com
This communication may contain privileged and/or confidential information. It is intended solely for the use of the addressee. If you are not the intended recipient, you are strictly prohibited from disclosing, copying, distributing or using any of this information. If you received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. Confidential, proprietary or time-sensitive communications should not be transmitted via the Internet, as there can be no assurance of actual or timely delivery, receipt and/or confidentiality. This is not an offer, or solicitation of any offer to buy or sell any security, investment or other product.
That is what I described. re-wording:
a0a should consist of four ports on each controller: e0e, e0f, e0g & e0h
e0e/e0g are on the same ASIC. They should go to switch #1, if possible, different blades on the same switch
e0f/e0h are on the same ASIC. They should go to switch #2, if possible, different blades on the same switch
be use to use mode active on the switch (to force LACP) --tmac
*Tim McCarthy* *Principal Consultant*
On Thu, Apr 9, 2015 at 11:46 AM, Sayla, Mustafa <MSAYLA@mesirowfinancial.com
wrote:
Thank you for your response. I verified we have the X6599A SFP. Just to clarify what I meant to ask is that can we have two ports go to one pair of Catalyst 6500 switch and 2 ports go to another pair of Catalyst 6500 switch and tie all four ports into one interface group so I have 40GB and also have switch redundancy. If not possible what is the best configuration I can have to have 40GB throughput.
Mustafa Sayla
*From:* toasters-bounces@teaparty.net [mailto: toasters-bounces@teaparty.net] *On Behalf Of *tmac *Sent:* Thursday, April 09, 2015 7:52 AM *To:* Clark, André M.
*Cc:* toasters@teaparty.net *Subject:* Re: FAS8040 network configuration in CDOT
Yeah, keep looking back at the FAS8020 for some reason...only has the four ports.
Make sure you have the X6599A (the 10-GIG SFP) and not the X6596(FC SFP).
For each controller:
a0a should consist of
e0e/e0g which should go to the same switch, if possible, different blades on the same switch and
e0f/e0h which should go to the same switch, if possible, different blades on the same switch
--> if you cannot split across blades, at least split across ASICs on the same blade.
--tmac
*Tim McCarthy*
*Principal Consultant*
On Thu, Apr 9, 2015 at 8:24 AM, Clark, André M. Andre.Clark@redeight.com wrote:
Each 8040/60/80 controller has 4 10GbE ports (e0a-e0d) and 4 UTA2 ports (e0e-e0h). Ports e0a-e0d, in a cDOT configuration, are used for the private cluster network while ports e0e-e0h are used for data communications. No extra 10G card is needed at this time.
Regards,
André M. Clark Andre.Clark@redeight.com | Sr. Consulting Architect | 917.388.8236
*Tell me I will forget... Show me I may remember... Involve me I WILL UNDERSTAND!!! Start by doing what's necessary, then what's possible, and suddenly you are doing the impossible!!!*
*Red8*, An Insight Investments Company
www.redeight.com
*From:* toasters-bounces@teaparty.net [mailto: toasters-bounces@teaparty.net] *On Behalf Of *tmac *Sent:* Thursday, 9 April, 2015 06:22 *To:* Francis Kim *Cc:* toasters@teaparty.net *Subject:* Re: FAS8040 network configuration in CDOT
No that won't work.
If you are configuring cDOT, at least two ports per controller are reserved for the cluster network (e0a & e0c)
This leaves e0b and e0d available for normal network access
The recommended configuration is to use all four ports for the cluster network (e0a, e0b, e0c & e0d).
In this case, if you need/want 10gig network connections you would need an extra 10gig card
AS indicated earlier in this thread, from each controller:
a0a should be a multimode_lacp VIF and contain:
e0b -> switch 1
e0d -> switch 2
Then you can create your VLANs: a0a-123, a0a-456
Place your LIFs on the tagged VLANs
Create a failover group for node-01:a0a-123 and node-02:a0a-123
Create a failover group for node-01:a0a-456 and node-02:a0a-456
Assign the failover group to the appropriate LIF
PS: DO NOT FORGET to disable spanning tree on the switch ports.
--> spanning tree protocol should only be enabled on ports connected to other switches.
--> it is not needed on ports connected to en hosts
--> in a cluster failover event it could take an extra 45 seconds for a port to become available.(not good)
--> remember to use the right version of the command. if it is a trunk, you usually need that on the line
--tmac
*Tim McCarthy*
*Principal Consultant*
On Wed, Apr 8, 2015 at 11:36 PM, Francis Kim fkim@berkcom.com wrote:
You can also tie the four links into a single interface group and carve it up into multiple VLANs.
What ports are you going to use for the cluster interconnect? I assume you're ordering a switchless cluster of two nodes?
*Francis Kim *| Engineer
510-644-1599 x334 | fkim@berkcom.com
*BerkCom* | www.berkcom.com
NetApp | Cisco | Supermicro | Brocade | VMware
On Apr 8, 2015, at 3:51 PM, Sayla, Mustafa MSAYLA@mesirowfinancial.com wrote:
We are looking to buy new FAS8040 and configure it with CDOT. I am looking for network recommendation with Cisco 6509 VSS. 8040 has four 10G ports per controller and I want to use all four ports to create an interface group with two VLAN, using VLAN tagging. I want 2 ports to go to one VSS switch and 2 ports to go to other VSS switch so I have switch redundancy also. I am not sure if it is doable since I am going to two different switches. Any recommendation is appreciated.
Mustafa
*Visit us on the Web at mesirowfinancial.com http://mesirowfinancial.com/*
*This communication may contain privileged and/or confidential information. It is intended solely for the use of the addressee. If you are not the intended recipient, you are strictly prohibited from disclosing, copying, distributing or using any of this information. If you received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. Confidential, proprietary or time-sensitive communications should not be transmitted via the Internet, as there can be no assurance of actual or timely delivery, receipt and/or confidentiality. This is not an offer, or solicitation of any offer to buy or sell any security, investment or other product.*_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
*Visit us on the Web at mesirowfinancial.com http://mesirowfinancial.com*
*This communication may contain privileged and/or confidential information. It is intended solely for the use of the addressee. If you are not the intended recipient, you are strictly prohibited from disclosing, copying, distributing or using any of this information. If you received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. Confidential, proprietary or time-sensitive communications should not be transmitted via the Internet, as there can be no assurance of actual or timely delivery, receipt and/or confidentiality. This is not an offer, or solicitation of any offer to buy or sell any security, investment or other product.*
e0e/e0g on the same ASIC? Not e0e/e0f on the same ASIC?
.
On Apr 9, 2015, at 5:51 AM, tmac <tmacmd@gmail.commailto:tmacmd@gmail.com> wrote:
That is what I described. re-wording:
a0a should consist of four ports on each controller: e0e, e0f, e0g & e0h
e0e/e0g are on the same ASIC. They should go to switch #1, if possible, different blades on the same switch e0f/e0h are on the same ASIC. They should go to switch #2, if possible, different blades on the same switch
be use to use mode active on the switch (to force LACP) --tmac
Tim McCarthy Principal Consultant
On Thu, Apr 9, 2015 at 11:46 AM, Sayla, Mustafa <MSAYLA@mesirowfinancial.commailto:MSAYLA@mesirowfinancial.com> wrote: Thank you for your response. I verified we have the X6599A SFP. Just to clarify what I meant to ask is that can we have two ports go to one pair of Catalyst 6500 switch and 2 ports go to another pair of Catalyst 6500 switch and tie all four ports into one interface group so I have 40GB and also have switch redundancy. If not possible what is the best configuration I can have to have 40GB throughput.
Mustafa Sayla
From: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net] On Behalf Of tmac Sent: Thursday, April 09, 2015 7:52 AM To: Clark, André M.
Cc: toasters@teaparty.netmailto:toasters@teaparty.net Subject: Re: FAS8040 network configuration in CDOT
Yeah, keep looking back at the FAS8020 for some reason...only has the four ports.
Make sure you have the X6599A (the 10-GIG SFP) and not the X6596(FC SFP). For each controller: a0a should consist of e0e/e0g which should go to the same switch, if possible, different blades on the same switch and e0f/e0h which should go to the same switch, if possible, different blades on the same switch --> if you cannot split across blades, at least split across ASICs on the same blade.
--tmac
Tim McCarthy Principal Consultant
On Thu, Apr 9, 2015 at 8:24 AM, Clark, André M. <Andre.Clark@redeight.commailto:Andre.Clark@redeight.com> wrote: Each 8040/60/80 controller has 4 10GbE ports (e0a-e0d) and 4 UTA2 ports (e0e-e0h). Ports e0a-e0d, in a cDOT configuration, are used for the private cluster network while ports e0e-e0h are used for data communications. No extra 10G card is needed at this time.
Regards,
André M. Clarkmailto:Andre.Clark@redeight.com | Sr. Consulting Architect | 917.388.8236tel:917.388.8236 Tell me I will forget... Show me I may remember... Involve me I WILL UNDERSTAND!!! Start by doing what's necessary, then what's possible, and suddenly you are doing the impossible!!! Red8, An Insight Investments Company www.redeight.comhttp://www.redeight.com/
From: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net] On Behalf Of tmac Sent: Thursday, 9 April, 2015 06:22 To: Francis Kim Cc: toasters@teaparty.netmailto:toasters@teaparty.net Subject: Re: FAS8040 network configuration in CDOT
No that won't work. If you are configuring cDOT, at least two ports per controller are reserved for the cluster network (e0a & e0c) This leaves e0b and e0d available for normal network access
The recommended configuration is to use all four ports for the cluster network (e0a, e0b, e0c & e0d). In this case, if you need/want 10gig network connections you would need an extra 10gig card
AS indicated earlier in this thread, from each controller: a0a should be a multimode_lacp VIF and contain: e0b -> switch 1 e0d -> switch 2
Then you can create your VLANs: a0a-123, a0a-456
Place your LIFs on the tagged VLANs
Create a failover group for node-01:a0a-123 and node-02:a0a-123 Create a failover group for node-01:a0a-456 and node-02:a0a-456
Assign the failover group to the appropriate LIF
PS: DO NOT FORGET to disable spanning tree on the switch ports. --> spanning tree protocol should only be enabled on ports connected to other switches. --> it is not needed on ports connected to en hosts --> in a cluster failover event it could take an extra 45 seconds for a port to become available.(not good) --> remember to use the right version of the command. if it is a trunk, you usually need that on the line
--tmac
Tim McCarthy Principal Consultant
On Wed, Apr 8, 2015 at 11:36 PM, Francis Kim <fkim@berkcom.commailto:fkim@berkcom.com> wrote: You can also tie the four links into a single interface group and carve it up into multiple VLANs.
What ports are you going to use for the cluster interconnect? I assume you’re ordering a switchless cluster of two nodes?
Francis Kim | Engineer 510-644-1599 x334tel:510-644-1599%20x334 | fkim@berkcom.commailto:fkim@berkcom.com
BerkCom | www.berkcom.comhttp://www.berkcom.com/ NetApp | Cisco | Supermicro | Brocade | VMware
On Apr 8, 2015, at 3:51 PM, Sayla, Mustafa <MSAYLA@mesirowfinancial.commailto:MSAYLA@mesirowfinancial.com> wrote:
We are looking to buy new FAS8040 and configure it with CDOT. I am looking for network recommendation with Cisco 6509 VSS. 8040 has four 10G ports per controller and I want to use all four ports to create an interface group with two VLAN, using VLAN tagging. I want 2 ports to go to one VSS switch and 2 ports to go to other VSS switch so I have switch redundancy also. I am not sure if it is doable since I am going to two different switches. Any recommendation is appreciated.
Mustafa Visit us on the Web at mesirowfinancial.comhttp://mesirowfinancial.com/ This communication may contain privileged and/or confidential information. It is intended solely for the use of the addressee. If you are not the intended recipient, you are strictly prohibited from disclosing, copying, distributing or using any of this information. If you received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. Confidential, proprietary or time-sensitive communications should not be transmitted via the Internet, as there can be no assurance of actual or timely delivery, receipt and/or confidentiality. This is not an offer, or solicitation of any offer to buy or sell any security, investment or other product._______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Visit us on the Web at mesirowfinancial.comhttp://mesirowfinancial.com
This communication may contain privileged and/or confidential information. It is intended solely for the use of the addressee. If you are not the intended recipient, you are strictly prohibited from disclosing, copying, distributing or using any of this information. If you received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. Confidential, proprietary or time-sensitive communications should not be transmitted via the Internet, as there can be no assurance of actual or timely delivery, receipt and/or confidentiality. This is not an offer, or solicitation of any offer to buy or sell any security, investment or other product.
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Yeah, copy/paste error. I knew what I wanted but did not proof read. thanks for the catch.
e0e/e0g are NOT on the same ASIC. They should go to switch #1, if possible, different blades on the same switch
e0f/e0h are NOT on the same ASIC. They should go to switch #2, if possible, different blades on the same switch
--tmac
*Tim McCarthy* *Principal Consultant*
On Thu, Apr 9, 2015 at 3:47 PM, Francis Kim fkim@berkcom.com wrote:
e0e/e0g on the same ASIC? Not e0e/e0f on the same ASIC?
.
On Apr 9, 2015, at 5:51 AM, tmac tmacmd@gmail.com wrote:
That is what I described. re-wording:
a0a should consist of four ports on each controller: e0e, e0f, e0g & e0h
e0e/e0g are on the same ASIC. They should go to switch #1, if possible, different blades on the same switch
e0f/e0h are on the same ASIC. They should go to switch #2, if possible, different blades on the same switch
be use to use mode active on the switch (to force LACP) --tmac
*Tim McCarthy* *Principal Consultant*
On Thu, Apr 9, 2015 at 11:46 AM, Sayla, Mustafa < MSAYLA@mesirowfinancial.com> wrote:
Thank you for your response. I verified we have the X6599A SFP. Just to clarify what I meant to ask is that can we have two ports go to one pair of Catalyst 6500 switch and 2 ports go to another pair of Catalyst 6500 switch and tie all four ports into one interface group so I have 40GB and also have switch redundancy. If not possible what is the best configuration I can have to have 40GB throughput.
Mustafa Sayla
*From:* toasters-bounces@teaparty.net [mailto: toasters-bounces@teaparty.net] *On Behalf Of *tmac *Sent:* Thursday, April 09, 2015 7:52 AM *To:* Clark, André M.
*Cc:* toasters@teaparty.net *Subject:* Re: FAS8040 network configuration in CDOT
Yeah, keep looking back at the FAS8020 for some reason...only has the four ports.
Make sure you have the X6599A (the 10-GIG SFP) and not the X6596(FC SFP).
For each controller:
a0a should consist of
e0e/e0g which should go to the same switch, if possible, different blades on the same switch and
e0f/e0h which should go to the same switch, if possible, different blades on the same switch
--> if you cannot split across blades, at least split across ASICs on the same blade.
--tmac
*Tim McCarthy*
*Principal Consultant*
On Thu, Apr 9, 2015 at 8:24 AM, Clark, André M. Andre.Clark@redeight.com wrote:
Each 8040/60/80 controller has 4 10GbE ports (e0a-e0d) and 4 UTA2 ports (e0e-e0h). Ports e0a-e0d, in a cDOT configuration, are used for the private cluster network while ports e0e-e0h are used for data communications. No extra 10G card is needed at this time.
Regards,
André M. Clark Andre.Clark@redeight.com | Sr. Consulting Architect | 917.388.8236
*Tell me I will forget... Show me I may remember... Involve me I WILL UNDERSTAND!!! Start by doing what's necessary, then what's possible, and suddenly you are doing the impossible!!!*
*Red8*, An Insight Investments Company
www.redeight.com
*From:* toasters-bounces@teaparty.net [mailto: toasters-bounces@teaparty.net] *On Behalf Of *tmac *Sent:* Thursday, 9 April, 2015 06:22 *To:* Francis Kim *Cc:* toasters@teaparty.net *Subject:* Re: FAS8040 network configuration in CDOT
No that won't work.
If you are configuring cDOT, at least two ports per controller are reserved for the cluster network (e0a & e0c)
This leaves e0b and e0d available for normal network access
The recommended configuration is to use all four ports for the cluster network (e0a, e0b, e0c & e0d).
In this case, if you need/want 10gig network connections you would need an extra 10gig card
AS indicated earlier in this thread, from each controller:
a0a should be a multimode_lacp VIF and contain:
e0b -> switch 1
e0d -> switch 2
Then you can create your VLANs: a0a-123, a0a-456
Place your LIFs on the tagged VLANs
Create a failover group for node-01:a0a-123 and node-02:a0a-123
Create a failover group for node-01:a0a-456 and node-02:a0a-456
Assign the failover group to the appropriate LIF
PS: DO NOT FORGET to disable spanning tree on the switch ports.
--> spanning tree protocol should only be enabled on ports connected to other switches.
--> it is not needed on ports connected to en hosts
--> in a cluster failover event it could take an extra 45 seconds for a port to become available.(not good)
--> remember to use the right version of the command. if it is a trunk, you usually need that on the line
--tmac
*Tim McCarthy*
*Principal Consultant*
On Wed, Apr 8, 2015 at 11:36 PM, Francis Kim fkim@berkcom.com wrote:
You can also tie the four links into a single interface group and carve it up into multiple VLANs.
What ports are you going to use for the cluster interconnect? I assume you're ordering a switchless cluster of two nodes?
*Francis Kim *| Engineer
510-644-1599 x334 | fkim@berkcom.com
*BerkCom* | www.berkcom.com
NetApp | Cisco | Supermicro | Brocade | VMware
On Apr 8, 2015, at 3:51 PM, Sayla, Mustafa MSAYLA@mesirowfinancial.com wrote:
We are looking to buy new FAS8040 and configure it with CDOT. I am looking for network recommendation with Cisco 6509 VSS. 8040 has four 10G ports per controller and I want to use all four ports to create an interface group with two VLAN, using VLAN tagging. I want 2 ports to go to one VSS switch and 2 ports to go to other VSS switch so I have switch redundancy also. I am not sure if it is doable since I am going to two different switches. Any recommendation is appreciated.
Mustafa
*Visit us on the Web at mesirowfinancial.com http://mesirowfinancial.com/*
*This communication may contain privileged and/or confidential information. It is intended solely for the use of the addressee. If you are not the intended recipient, you are strictly prohibited from disclosing, copying, distributing or using any of this information. If you received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. Confidential, proprietary or time-sensitive communications should not be transmitted via the Internet, as there can be no assurance of actual or timely delivery, receipt and/or confidentiality. This is not an offer, or solicitation of any offer to buy or sell any security, investment or other product.*_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
*Visit us on the Web at mesirowfinancial.com*This communication may contain privileged and/or confidential information. It is intended solely for the use of the addressee. If you are not the intended recipient, you are strictly prohibited from disclosing, copying, distributing or using any of this information. If you received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. Confidential, proprietary or time-sensitive communications should not be transmitted via the Internet, as there can be no assurance of actual or timely delivery, receipt and/or confidentiality. This is not an offer, or solicitation of any offer to buy or sell any security, investment or other product.*
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
You're welcome.
.
On Apr 9, 2015, at 9:50 AM, tmac <tmacmd@gmail.commailto:tmacmd@gmail.com> wrote:
Yeah, copy/paste error. I knew what I wanted but did not proof read. thanks for the catch.
e0e/e0g are NOT on the same ASIC. They should go to switch #1, if possible, different blades on the same switch e0f/e0h are NOT on the same ASIC. They should go to switch #2, if possible, different blades on the same switch
--tmac
Tim McCarthy Principal Consultant
On Thu, Apr 9, 2015 at 3:47 PM, Francis Kim <fkim@berkcom.commailto:fkim@berkcom.com> wrote: e0e/e0g on the same ASIC? Not e0e/e0f on the same ASIC?
.
On Apr 9, 2015, at 5:51 AM, tmac <tmacmd@gmail.commailto:tmacmd@gmail.com> wrote:
That is what I described. re-wording:
a0a should consist of four ports on each controller: e0e, e0f, e0g & e0h
e0e/e0g are on the same ASIC. They should go to switch #1, if possible, different blades on the same switch e0f/e0h are on the same ASIC. They should go to switch #2, if possible, different blades on the same switch
be use to use mode active on the switch (to force LACP) --tmac
Tim McCarthy Principal Consultant
On Thu, Apr 9, 2015 at 11:46 AM, Sayla, Mustafa <MSAYLA@mesirowfinancial.commailto:MSAYLA@mesirowfinancial.com> wrote: Thank you for your response. I verified we have the X6599A SFP. Just to clarify what I meant to ask is that can we have two ports go to one pair of Catalyst 6500 switch and 2 ports go to another pair of Catalyst 6500 switch and tie all four ports into one interface group so I have 40GB and also have switch redundancy. If not possible what is the best configuration I can have to have 40GB throughput.
Mustafa Sayla
From: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net] On Behalf Of tmac Sent: Thursday, April 09, 2015 7:52 AM To: Clark, André M.
Cc: toasters@teaparty.netmailto:toasters@teaparty.net Subject: Re: FAS8040 network configuration in CDOT
Yeah, keep looking back at the FAS8020 for some reason...only has the four ports.
Make sure you have the X6599A (the 10-GIG SFP) and not the X6596(FC SFP). For each controller: a0a should consist of e0e/e0g which should go to the same switch, if possible, different blades on the same switch and e0f/e0h which should go to the same switch, if possible, different blades on the same switch --> if you cannot split across blades, at least split across ASICs on the same blade.
--tmac
Tim McCarthy Principal Consultant
On Thu, Apr 9, 2015 at 8:24 AM, Clark, André M. <Andre.Clark@redeight.commailto:Andre.Clark@redeight.com> wrote: Each 8040/60/80 controller has 4 10GbE ports (e0a-e0d) and 4 UTA2 ports (e0e-e0h). Ports e0a-e0d, in a cDOT configuration, are used for the private cluster network while ports e0e-e0h are used for data communications. No extra 10G card is needed at this time.
Regards,
André M. Clarkmailto:Andre.Clark@redeight.com | Sr. Consulting Architect | 917.388.8236tel:917.388.8236 Tell me I will forget... Show me I may remember... Involve me I WILL UNDERSTAND!!! Start by doing what's necessary, then what's possible, and suddenly you are doing the impossible!!! Red8, An Insight Investments Company www.redeight.comhttp://www.redeight.com/
From: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net] On Behalf Of tmac Sent: Thursday, 9 April, 2015 06:22 To: Francis Kim Cc: toasters@teaparty.netmailto:toasters@teaparty.net Subject: Re: FAS8040 network configuration in CDOT
No that won't work. If you are configuring cDOT, at least two ports per controller are reserved for the cluster network (e0a & e0c) This leaves e0b and e0d available for normal network access
The recommended configuration is to use all four ports for the cluster network (e0a, e0b, e0c & e0d). In this case, if you need/want 10gig network connections you would need an extra 10gig card
AS indicated earlier in this thread, from each controller: a0a should be a multimode_lacp VIF and contain: e0b -> switch 1 e0d -> switch 2
Then you can create your VLANs: a0a-123, a0a-456
Place your LIFs on the tagged VLANs
Create a failover group for node-01:a0a-123 and node-02:a0a-123 Create a failover group for node-01:a0a-456 and node-02:a0a-456
Assign the failover group to the appropriate LIF
PS: DO NOT FORGET to disable spanning tree on the switch ports. --> spanning tree protocol should only be enabled on ports connected to other switches. --> it is not needed on ports connected to en hosts --> in a cluster failover event it could take an extra 45 seconds for a port to become available.(not good) --> remember to use the right version of the command. if it is a trunk, you usually need that on the line
--tmac
Tim McCarthy Principal Consultant
On Wed, Apr 8, 2015 at 11:36 PM, Francis Kim <fkim@berkcom.commailto:fkim@berkcom.com> wrote: You can also tie the four links into a single interface group and carve it up into multiple VLANs.
What ports are you going to use for the cluster interconnect? I assume you’re ordering a switchless cluster of two nodes?
Francis Kim | Engineer 510-644-1599 x334tel:510-644-1599%20x334 | fkim@berkcom.commailto:fkim@berkcom.com
BerkCom | www.berkcom.comhttp://www.berkcom.com/ NetApp | Cisco | Supermicro | Brocade | VMware
On Apr 8, 2015, at 3:51 PM, Sayla, Mustafa <MSAYLA@mesirowfinancial.commailto:MSAYLA@mesirowfinancial.com> wrote:
We are looking to buy new FAS8040 and configure it with CDOT. I am looking for network recommendation with Cisco 6509 VSS. 8040 has four 10G ports per controller and I want to use all four ports to create an interface group with two VLAN, using VLAN tagging. I want 2 ports to go to one VSS switch and 2 ports to go to other VSS switch so I have switch redundancy also. I am not sure if it is doable since I am going to two different switches. Any recommendation is appreciated.
Mustafa Visit us on the Web at mesirowfinancial.comhttp://mesirowfinancial.com/ This communication may contain privileged and/or confidential information. It is intended solely for the use of the addressee. If you are not the intended recipient, you are strictly prohibited from disclosing, copying, distributing or using any of this information. If you received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. Confidential, proprietary or time-sensitive communications should not be transmitted via the Internet, as there can be no assurance of actual or timely delivery, receipt and/or confidentiality. This is not an offer, or solicitation of any offer to buy or sell any security, investment or other product._______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Visit us on the Web at mesirowfinancial.comhttp://mesirowfinancial.com
This communication may contain privileged and/or confidential information. It is intended solely for the use of the addressee. If you are not the intended recipient, you are strictly prohibited from disclosing, copying, distributing or using any of this information. If you received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. Confidential, proprietary or time-sensitive communications should not be transmitted via the Internet, as there can be no assurance of actual or timely delivery, receipt and/or confidentiality. This is not an offer, or solicitation of any offer to buy or sell any security, investment or other product.
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Sayla,
Yes, you can create a four port interface group on your FAS8040 with two ports going to each of your 6500 switches.
Some things to consider:
1. It is recommended you pick interface group ports from network adapters of the same model. Otherwise, certain features (TSO, LRO, checksum offloading, etc) may be disabled for the interface group if member ports don’t have identical support for those features. If you want to mix ports from 10GbE adapters with onboard ports, check with NGS before proceeding. 2. Maximum of 4 ports in an interface group if using 10gbE. 3. Of the multiple flavors of interface grouping (Single, Multimode, Dynamic Multimode), Dynamic Multimode is the most commonly used. 4. Several load balancing schemes are available for the interface group ports, but IP address is the most commonly used. If you were hoping to see a single 40GbE pipe, then Round-robin load balancing is the way to go, but that’s for a VERY limited use case where TCP endpoints can handle out-of-order packet delivery. 5. Be aware that load balancing is done only on outbound traffic from your FAS controller. Make sure the load balancing algorithm setting on your switches’ port channel group is consistent with your interface group load balancing scheme.
Further comments about the private cluster network. While four ports are recommended, two ports are just fine most of the time for a two-node CDOT, especially in a switchless configuration. In fact, the latest Data_ONTAP_83_Express_Setup_Guide_for_80xx_Systems (https://www.dropbox.com/s/azuo4rdglm3qg9n/Data_ONTAP_83_Express_Setup_Guide_...) shows you how.
Typical clusters show tens of KB/s on the cluster-interconnect during most operations when LIFs are on the same node as the volumes (and ultimately the aggregates) they are serving. Cluster ring database updates don’t push that much data. Even when LIFs get migrated off-node and data is served out indirect, two 10GbE ports in your cluster interconnect should give you enough bandwidth to handle this. If you intend to not manage LIF/volume residency at all and let LIFs and volumes fall on whatever node they may, then additional cluster interconnect links will buy you some protection. Otherwise, going four links for cluster interconnect will pay off the most if you do frequent volume relocates across nodes, or once your cluster grows with additional node pairs. Francis Kim | Engineer 510-644-1599 x334 | fkim@berkcom.commailto:fkim@berkcom.com
BerkCom | www.berkcom.comhttp://www.berkcom.com/ NetApp | Cisco | Supermicro | Brocade | VMware
On Apr 9, 2015, at 5:46 AM, Sayla, Mustafa <MSAYLA@mesirowfinancial.commailto:MSAYLA@mesirowfinancial.com> wrote:
Thank you for your response. I verified we have the X6599A SFP. Just to clarify what I meant to ask is that can we have two ports go to one pair of Catalyst 6500 switch and 2 ports go to another pair of Catalyst 6500 switch and tie all four ports into one interface group so I have 40GB and also have switch redundancy. If not possible what is the best configuration I can have to have 40GB throughput.
Mustafa Sayla
From: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of tmac Sent: Thursday, April 09, 2015 7:52 AM To: Clark, André M. Cc: toasters@teaparty.netmailto:toasters@teaparty.net Subject: Re: FAS8040 network configuration in CDOT
Yeah, keep looking back at the FAS8020 for some reason...only has the four ports.
Make sure you have the X6599A (the 10-GIG SFP) and not the X6596(FC SFP). For each controller: a0a should consist of e0e/e0g which should go to the same switch, if possible, different blades on the same switch and e0f/e0h which should go to the same switch, if possible, different blades on the same switch --> if you cannot split across blades, at least split across ASICs on the same blade.
--tmac
Tim McCarthy Principal Consultant
On Thu, Apr 9, 2015 at 8:24 AM, Clark, André M. <Andre.Clark@redeight.commailto:Andre.Clark@redeight.com> wrote: Each 8040/60/80 controller has 4 10GbE ports (e0a-e0d) and 4 UTA2 ports (e0e-e0h). Ports e0a-e0d, in a cDOT configuration, are used for the private cluster network while ports e0e-e0h are used for data communications. No extra 10G card is needed at this time.
Regards,
André M. Clarkmailto:Andre.Clark@redeight.com | Sr. Consulting Architect | 917.388.8236tel:917.388.8236 Tell me I will forget... Show me I may remember... Involve me I WILL UNDERSTAND!!! Start by doing what's necessary, then what's possible, and suddenly you are doing the impossible!!! Red8, An Insight Investments Company www.redeight.comhttp://www.redeight.com/
From: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net] On Behalf Of tmac Sent: Thursday, 9 April, 2015 06:22 To: Francis Kim Cc: toasters@teaparty.netmailto:toasters@teaparty.net Subject: Re: FAS8040 network configuration in CDOT
No that won't work. If you are configuring cDOT, at least two ports per controller are reserved for the cluster network (e0a & e0c) This leaves e0b and e0d available for normal network access
The recommended configuration is to use all four ports for the cluster network (e0a, e0b, e0c & e0d). In this case, if you need/want 10gig network connections you would need an extra 10gig card
AS indicated earlier in this thread, from each controller: a0a should be a multimode_lacp VIF and contain: e0b -> switch 1 e0d -> switch 2
Then you can create your VLANs: a0a-123, a0a-456
Place your LIFs on the tagged VLANs
Create a failover group for node-01:a0a-123 and node-02:a0a-123 Create a failover group for node-01:a0a-456 and node-02:a0a-456
Assign the failover group to the appropriate LIF
PS: DO NOT FORGET to disable spanning tree on the switch ports. --> spanning tree protocol should only be enabled on ports connected to other switches. --> it is not needed on ports connected to en hosts --> in a cluster failover event it could take an extra 45 seconds for a port to become available.(not good) --> remember to use the right version of the command. if it is a trunk, you usually need that on the line
--tmac
Tim McCarthy Principal Consultant
On Wed, Apr 8, 2015 at 11:36 PM, Francis Kim <fkim@berkcom.commailto:fkim@berkcom.com> wrote: You can also tie the four links into a single interface group and carve it up into multiple VLANs.
What ports are you going to use for the cluster interconnect? I assume you’re ordering a switchless cluster of two nodes?
Francis Kim | Engineer 510-644-1599 x334tel:510-644-1599%20x334 | fkim@berkcom.commailto:fkim@berkcom.com
BerkCom | www.berkcom.comhttp://www.berkcom.com/ NetApp | Cisco | Supermicro | Brocade | VMware
On Apr 8, 2015, at 3:51 PM, Sayla, Mustafa <MSAYLA@mesirowfinancial.commailto:MSAYLA@mesirowfinancial.com> wrote:
We are looking to buy new FAS8040 and configure it with CDOT. I am looking for network recommendation with Cisco 6509 VSS. 8040 has four 10G ports per controller and I want to use all four ports to create an interface group with two VLAN, using VLAN tagging. I want 2 ports to go to one VSS switch and 2 ports to go to other VSS switch so I have switch redundancy also. I am not sure if it is doable since I am going to two different switches. Any recommendation is appreciated.
Mustafa Visit us on the Web at mesirowfinancial.comhttp://mesirowfinancial.com/ This communication may contain privileged and/or confidential information. It is intended solely for the use of the addressee. If you are not the intended recipient, you are strictly prohibited from disclosing, copying, distributing or using any of this information. If you received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. Confidential, proprietary or time-sensitive communications should not be transmitted via the Internet, as there can be no assurance of actual or timely delivery, receipt and/or confidentiality. This is not an offer, or solicitation of any offer to buy or sell any security, investment or other product._______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Visit us on the Web at mesirowfinancial.comhttp://mesirowfinancial.com/
This communication may contain privileged and/or confidential information. It is intended solely for the use of the addressee. If you are not the intended recipient, you are strictly prohibited from disclosing, copying, distributing or using any of this information. If you received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. Confidential, proprietary or time-sensitive communications should not be transmitted via the Internet, as there can be no assurance of actual or timely delivery, receipt and/or confidentiality. This is not an offer, or solicitation of any offer to buy or sell any security, investment or other product._______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Interesting thread....
A peer suggested recently that in cDOT failover groups could be used instead of ifgrps for availability. It was suggested as it simplifies the configuration (don't have ifgrps and failover groups) and means a LIF can failover to any port in the cluster (depending on type of LIF of course) rather than just the ports on the node.
Not considering throughput has anyone else heard of or considered this configuration?
Thanks Martin
-- View this message in context: http://network-appliance-toasters.10978.n7.nabble.com/FAS8040-network-config... Sent from the Network Appliance - Toasters mailing list archive at Nabble.com.
Yes, using failover groups across an 8 node 6220 cluster, works very well, had quite a few fail overs in the last 2.5 years, and the system has maintained service during all of them, so well that the applications/users failed to notice anything happening, including a decent sized VMWare setup, getting it’s storage via NFS.
~Mark
mark.flint@sanger.ac.uk
On 14 Apr 2015, at 13:30, Martin martin@leggatt.me.uk wrote:
Interesting thread....
A peer suggested recently that in cDOT failover groups could be used instead of ifgrps for availability. It was suggested as it simplifies the configuration (don't have ifgrps and failover groups) and means a LIF can failover to any port in the cluster (depending on type of LIF of course) rather than just the ports on the node.
Not considering throughput has anyone else heard of or considered this configuration?
Thanks Martin
-- View this message in context: http://network-appliance-toasters.10978.n7.nabble.com/FAS8040-network-config... Sent from the Network Appliance - Toasters mailing list archive at Nabble.com. _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
We take advantage of this when we don't have enough network connectivity to ensure proper redunancy, but still use ifgrps whenever possible. It makes the load balancing less involved- if you use failover groups, you need to try and spread your LIFs around, but if you have an ifgrp, you just tie together all the physical interfaces and then make all your LIFs' home there.
On Tue, Apr 14, 2015 at 8:30 AM, Martin martin@leggatt.me.uk wrote:
Interesting thread....
A peer suggested recently that in cDOT failover groups could be used instead of ifgrps for availability. It was suggested as it simplifies the configuration (don't have ifgrps and failover groups) and means a LIF can failover to any port in the cluster (depending on type of LIF of course) rather than just the ports on the node.
Not considering throughput has anyone else heard of or considered this configuration?
Thanks Martin
-- View this message in context: http://network-appliance-toasters.10978.n7.nabble.com/FAS8040-network-config... Sent from the Network Appliance - Toasters mailing list archive at Nabble.com. _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Ok, let me date myself now...
Way back on GX (version 10) and the original cDOT of 8.0, there actually was no process to create ifgrps and the only choice you had was to do the failover groups...
I think 8.1 introduced LACP.
--tmac
*Tim McCarthy* *Principal Consultant*
On Tue, Apr 14, 2015 at 9:18 AM, Basil basilberntsen@gmail.com wrote:
We take advantage of this when we don't have enough network connectivity to ensure proper redunancy, but still use ifgrps whenever possible. It makes the load balancing less involved- if you use failover groups, you need to try and spread your LIFs around, but if you have an ifgrp, you just tie together all the physical interfaces and then make all your LIFs' home there.
On Tue, Apr 14, 2015 at 8:30 AM, Martin martin@leggatt.me.uk wrote:
Interesting thread....
A peer suggested recently that in cDOT failover groups could be used instead of ifgrps for availability. It was suggested as it simplifies the configuration (don't have ifgrps and failover groups) and means a LIF can failover to any port in the cluster (depending on type of LIF of course) rather than just the ports on the node.
Not considering throughput has anyone else heard of or considered this configuration?
Thanks Martin
-- View this message in context: http://network-appliance-toasters.10978.n7.nabble.com/FAS8040-network-config... Sent from the Network Appliance - Toasters mailing list archive at Nabble.com. _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
We do ifgrps within a node and then failover-groups between nodes. If one link goes down, traffic still stays local to the node. If the node goes down, traffic migrates as expected.
We also do a LIF per volume so that the LIF is local to the same node as the volume and a vol move can be done without having to remount the clients.
John
On Tue, Apr 14, 2015 at 09:18:30AM -0400, Basil wrote:
We take advantage of this when we don't have enough network connectivity to ensure proper redunancy, but still use ifgrps whenever possible. It makes the load balancing less involved- if you use failover groups, you need to try and spread your LIFs around, but if you have an ifgrp, you just tie together all the physical interfaces and then make all your LIFs' home there.
On Tue, Apr 14, 2015 at 8:30 AM, Martin martin@leggatt.me.uk wrote:
Interesting thread....
A peer suggested recently that in cDOT failover groups could be used instead of ifgrps for availability. It was suggested as it simplifies the configuration (don't have ifgrps and failover groups) and means a LIF can failover to any port in the cluster (depending on type of LIF of course) rather than just the ports on the node.
Not considering throughput has anyone else heard of or considered this configuration?
Thanks Martin
-- View this message in context: http://network-appliance-toasters.10978.n7.nabble.com/FAS8040-network-config... Sent from the Network Appliance - Toasters mailing list archive at Nabble.com.
Thanks guys, really useful, made things a lot clearer.
John your comment "do ifgrps within a node and then failover-groups between nodes. If one link goes down, traffic still stays local to the node. If the node goes down, traffic migrates as expected. " makes sense for SMB/CIFS environments (or I guess NFSv4).
Martin
-- View this message in context: http://network-appliance-toasters.10978.n7.nabble.com/FAS8040-network-config... Sent from the Network Appliance - Toasters mailing list archive at Nabble.com.
Actually, only makes sense for NFS/CIFS
Failover groups do not work with SAN protocols. cDOT will not even let you configure a failover group on an iSCSI of FCP LIF.
Also, not sure if you can put an iSCSI LIF on an ifgrp. (you probably can)
--tmac
*Tim McCarthy* *Principal Consultant*
On Wed, Apr 15, 2015 at 5:46 AM, Martin martin@leggatt.me.uk wrote:
Thanks guys, really useful, made things a lot clearer.
John your comment "do ifgrps within a node and then failover-groups between nodes. If one link goes down, traffic still stays local to the node. If the node goes down, traffic migrates as expected. " makes sense for SMB/CIFS environments (or I guess NFSv4).
Martin
-- View this message in context: http://network-appliance-toasters.10978.n7.nabble.com/FAS8040-network-config... Sent from the Network Appliance - Toasters mailing list archive at Nabble.com. _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Found this:
There are two methods to achieve path redundancy if using iSCSI (RFC3720) in clustered Data ONTAP: by using ifgrps to aggregate more than one physical port in partnership with an LACP-enabled switch, or by configuring hosts to use MPIO over multiple distinct physical links. Both of these methods allow a storage controller and host to use aggregated bandwidth and both can survive the failure of one of the paths from host to storage controller. However, MPIO is already a requirement for using block storage with clustered Data ONTAP, and using MPIO has the further advantage of no additional switch configuration or port trunking configuration being required. Also, using an ifgrp for path management when using iSCSI is not supported, again, you will want to use MPIO with iSCSI. But, using an ifgrp as a port for an iSCSI LIF is supported.
--tmac
*Tim McCarthy* *Principal Consultant*
On Wed, Apr 15, 2015 at 6:22 AM, tmac tmacmd@gmail.com wrote:
Actually, only makes sense for NFS/CIFS
Failover groups do not work with SAN protocols. cDOT will not even let you configure a failover group on an iSCSI of FCP LIF.
Also, not sure if you can put an iSCSI LIF on an ifgrp. (you probably can)
--tmac
*Tim McCarthy* *Principal Consultant*
On Wed, Apr 15, 2015 at 5:46 AM, Martin martin@leggatt.me.uk wrote:
Thanks guys, really useful, made things a lot clearer.
John your comment "do ifgrps within a node and then failover-groups between nodes. If one link goes down, traffic still stays local to the node. If the node goes down, traffic migrates as expected. " makes sense for SMB/CIFS environments (or I guess NFSv4).
Martin
-- View this message in context: http://network-appliance-toasters.10978.n7.nabble.com/FAS8040-network-config... Sent from the Network Appliance - Toasters mailing list archive at Nabble.com. _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Yep, I've put iSCSI LIFs on ifgrps plenty of times, more often than not actually. This way NFS/CIFS and iSCSI can share the same precious 10GbE physical ports and all can be optimally configured (just separate them with vlans. Please separate them with vlans). Less admin to build as well - one LIF per node is just fine, and you only need one session per node established from the host instead of two. But yes always, always use MPIO with iSCSI.
Peta
On 15 April 2015 at 21:05, tmac tmacmd@gmail.com wrote:
Found this:
There are two methods to achieve path redundancy if using iSCSI (RFC3720) in clustered Data ONTAP: by using ifgrps to aggregate more than one physical port in partnership with an LACP-enabled switch, or by configuring hosts to use MPIO over multiple distinct physical links. Both of these methods allow a storage controller and host to use aggregated bandwidth and both can survive the failure of one of the paths from host to storage controller. However, MPIO is already a requirement for using block storage with clustered Data ONTAP, and using MPIO has the further advantage of no additional switch configuration or port trunking configuration being required. Also, using an ifgrp for path management when using iSCSI is not supported, again, you will want to use MPIO with iSCSI. But, using an ifgrp as a port for an iSCSI LIF is supported.
--tmac
*Tim McCarthy* *Principal Consultant*
On Wed, Apr 15, 2015 at 6:22 AM, tmac tmacmd@gmail.com wrote:
Actually, only makes sense for NFS/CIFS
Failover groups do not work with SAN protocols. cDOT will not even let you configure a failover group on an iSCSI of FCP LIF.
Also, not sure if you can put an iSCSI LIF on an ifgrp. (you probably can)
--tmac
*Tim McCarthy* *Principal Consultant*
On Wed, Apr 15, 2015 at 5:46 AM, Martin martin@leggatt.me.uk wrote:
Thanks guys, really useful, made things a lot clearer.
John your comment "do ifgrps within a node and then failover-groups between nodes. If one link goes down, traffic still stays local to the node. If the node goes down, traffic migrates as expected. " makes sense for SMB/CIFS environments (or I guess NFSv4).
Martin
-- View this message in context: http://network-appliance-toasters.10978.n7.nabble.com/FAS8040-network-config... Sent from the Network Appliance - Toasters mailing list archive at Nabble.com. _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
The only downside from an availability perspective is that CIFS connection drops during a failover, whereas a link change in an ifgrp is transparent to CIFS. Of course this is a non-issue for NFS, at least for v3.
.
On Apr 14, 2015, at 5:44 AM, Martin martin@leggatt.me.uk wrote:
Interesting thread....
A peer suggested recently that in cDOT failover groups could be used instead of ifgrps for availability. It was suggested as it simplifies the configuration (don't have ifgrps and failover groups) and means a LIF can failover to any port in the cluster (depending on type of LIF of course) rather than just the ports on the node.
Not considering throughput has anyone else heard of or considered this configuration?
Thanks Martin
-- View this message in context: http://network-appliance-toasters.10978.n7.nabble.com/FAS8040-network-config... Sent from the Network Appliance - Toasters mailing list archive at Nabble.com. _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
-
Basil -
Francis Kim -
John Clear -
Mark Flint -
Martin -
Peta Thames -
Sayla, Mustafa -
tmac