Sayla,
Yes, you can create a four port interface group on your FAS8040 with two ports going to each of your 6500 switches.
Some things to consider:
1. It is recommended you pick interface group ports from network adapters of the same model. Otherwise, certain features (TSO, LRO, checksum offloading, etc) may be disabled for the interface group if member ports don’t have identical support for those features. If you want to mix ports from 10GbE adapters with onboard ports, check with NGS before proceeding. 2. Maximum of 4 ports in an interface group if using 10gbE. 3. Of the multiple flavors of interface grouping (Single, Multimode, Dynamic Multimode), Dynamic Multimode is the most commonly used. 4. Several load balancing schemes are available for the interface group ports, but IP address is the most commonly used. If you were hoping to see a single 40GbE pipe, then Round-robin load balancing is the way to go, but that’s for a VERY limited use case where TCP endpoints can handle out-of-order packet delivery. 5. Be aware that load balancing is done only on outbound traffic from your FAS controller. Make sure the load balancing algorithm setting on your switches’ port channel group is consistent with your interface group load balancing scheme.
Further comments about the private cluster network. While four ports are recommended, two ports are just fine most of the time for a two-node CDOT, especially in a switchless configuration. In fact, the latest Data_ONTAP_83_Express_Setup_Guide_for_80xx_Systems (https://www.dropbox.com/s/azuo4rdglm3qg9n/Data_ONTAP_83_Express_Setup_Guide_...) shows you how.
Typical clusters show tens of KB/s on the cluster-interconnect during most operations when LIFs are on the same node as the volumes (and ultimately the aggregates) they are serving. Cluster ring database updates don’t push that much data. Even when LIFs get migrated off-node and data is served out indirect, two 10GbE ports in your cluster interconnect should give you enough bandwidth to handle this. If you intend to not manage LIF/volume residency at all and let LIFs and volumes fall on whatever node they may, then additional cluster interconnect links will buy you some protection. Otherwise, going four links for cluster interconnect will pay off the most if you do frequent volume relocates across nodes, or once your cluster grows with additional node pairs. Francis Kim | Engineer 510-644-1599 x334 | fkim@berkcom.commailto:fkim@berkcom.com
BerkCom | www.berkcom.comhttp://www.berkcom.com/ NetApp | Cisco | Supermicro | Brocade | VMware
On Apr 9, 2015, at 5:46 AM, Sayla, Mustafa <MSAYLA@mesirowfinancial.commailto:MSAYLA@mesirowfinancial.com> wrote:
Thank you for your response. I verified we have the X6599A SFP. Just to clarify what I meant to ask is that can we have two ports go to one pair of Catalyst 6500 switch and 2 ports go to another pair of Catalyst 6500 switch and tie all four ports into one interface group so I have 40GB and also have switch redundancy. If not possible what is the best configuration I can have to have 40GB throughput.
Mustafa Sayla
From: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of tmac Sent: Thursday, April 09, 2015 7:52 AM To: Clark, André M. Cc: toasters@teaparty.netmailto:toasters@teaparty.net Subject: Re: FAS8040 network configuration in CDOT
Yeah, keep looking back at the FAS8020 for some reason...only has the four ports.
Make sure you have the X6599A (the 10-GIG SFP) and not the X6596(FC SFP). For each controller: a0a should consist of e0e/e0g which should go to the same switch, if possible, different blades on the same switch and e0f/e0h which should go to the same switch, if possible, different blades on the same switch --> if you cannot split across blades, at least split across ASICs on the same blade.
--tmac
Tim McCarthy Principal Consultant
On Thu, Apr 9, 2015 at 8:24 AM, Clark, André M. <Andre.Clark@redeight.commailto:Andre.Clark@redeight.com> wrote: Each 8040/60/80 controller has 4 10GbE ports (e0a-e0d) and 4 UTA2 ports (e0e-e0h). Ports e0a-e0d, in a cDOT configuration, are used for the private cluster network while ports e0e-e0h are used for data communications. No extra 10G card is needed at this time.
Regards,
André M. Clarkmailto:Andre.Clark@redeight.com | Sr. Consulting Architect | 917.388.8236tel:917.388.8236 Tell me I will forget... Show me I may remember... Involve me I WILL UNDERSTAND!!! Start by doing what's necessary, then what's possible, and suddenly you are doing the impossible!!! Red8, An Insight Investments Company www.redeight.comhttp://www.redeight.com/
From: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net] On Behalf Of tmac Sent: Thursday, 9 April, 2015 06:22 To: Francis Kim Cc: toasters@teaparty.netmailto:toasters@teaparty.net Subject: Re: FAS8040 network configuration in CDOT
No that won't work. If you are configuring cDOT, at least two ports per controller are reserved for the cluster network (e0a & e0c) This leaves e0b and e0d available for normal network access
The recommended configuration is to use all four ports for the cluster network (e0a, e0b, e0c & e0d). In this case, if you need/want 10gig network connections you would need an extra 10gig card
AS indicated earlier in this thread, from each controller: a0a should be a multimode_lacp VIF and contain: e0b -> switch 1 e0d -> switch 2
Then you can create your VLANs: a0a-123, a0a-456
Place your LIFs on the tagged VLANs
Create a failover group for node-01:a0a-123 and node-02:a0a-123 Create a failover group for node-01:a0a-456 and node-02:a0a-456
Assign the failover group to the appropriate LIF
PS: DO NOT FORGET to disable spanning tree on the switch ports. --> spanning tree protocol should only be enabled on ports connected to other switches. --> it is not needed on ports connected to en hosts --> in a cluster failover event it could take an extra 45 seconds for a port to become available.(not good) --> remember to use the right version of the command. if it is a trunk, you usually need that on the line
--tmac
Tim McCarthy Principal Consultant
On Wed, Apr 8, 2015 at 11:36 PM, Francis Kim <fkim@berkcom.commailto:fkim@berkcom.com> wrote: You can also tie the four links into a single interface group and carve it up into multiple VLANs.
What ports are you going to use for the cluster interconnect? I assume you’re ordering a switchless cluster of two nodes?
Francis Kim | Engineer 510-644-1599 x334tel:510-644-1599%20x334 | fkim@berkcom.commailto:fkim@berkcom.com
BerkCom | www.berkcom.comhttp://www.berkcom.com/ NetApp | Cisco | Supermicro | Brocade | VMware
On Apr 8, 2015, at 3:51 PM, Sayla, Mustafa <MSAYLA@mesirowfinancial.commailto:MSAYLA@mesirowfinancial.com> wrote:
We are looking to buy new FAS8040 and configure it with CDOT. I am looking for network recommendation with Cisco 6509 VSS. 8040 has four 10G ports per controller and I want to use all four ports to create an interface group with two VLAN, using VLAN tagging. I want 2 ports to go to one VSS switch and 2 ports to go to other VSS switch so I have switch redundancy also. I am not sure if it is doable since I am going to two different switches. Any recommendation is appreciated.
Mustafa Visit us on the Web at mesirowfinancial.comhttp://mesirowfinancial.com/ This communication may contain privileged and/or confidential information. It is intended solely for the use of the addressee. If you are not the intended recipient, you are strictly prohibited from disclosing, copying, distributing or using any of this information. If you received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. Confidential, proprietary or time-sensitive communications should not be transmitted via the Internet, as there can be no assurance of actual or timely delivery, receipt and/or confidentiality. This is not an offer, or solicitation of any offer to buy or sell any security, investment or other product._______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Visit us on the Web at mesirowfinancial.comhttp://mesirowfinancial.com/
This communication may contain privileged and/or confidential information. It is intended solely for the use of the addressee. If you are not the intended recipient, you are strictly prohibited from disclosing, copying, distributing or using any of this information. If you received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. Confidential, proprietary or time-sensitive communications should not be transmitted via the Internet, as there can be no assurance of actual or timely delivery, receipt and/or confidentiality. This is not an offer, or solicitation of any offer to buy or sell any security, investment or other product._______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters