Sayla,

Yes, you can create a four port interface group on your FAS8040 with two ports going to each of your 6500 switches.

Some things to consider:

1. It is recommended you pick interface group ports from network adapters of the same model.  Otherwise, certain features (TSO, LRO, checksum offloading, etc) may be disabled for the interface group if member ports don’t have identical support for those features.  If you want to mix ports from 10GbE adapters with onboard ports, check with NGS before proceeding.
2. Maximum of 4 ports in an interface group if using 10gbE.
3. Of the multiple flavors of interface grouping (Single, Multimode, Dynamic Multimode), Dynamic Multimode is the most commonly used.
4. Several load balancing schemes are available for the interface group ports, but IP address is the most commonly used.  If you were hoping to see a single 40GbE pipe, then Round-robin load balancing is the way to go, but that’s for a VERY limited use case where TCP endpoints can handle out-of-order packet delivery.
5. Be aware that load balancing is done only on outbound traffic from your FAS controller.  Make sure the load balancing algorithm setting on your switches’ port channel group is consistent with your interface group load balancing scheme.

Further comments about the private cluster network.  While four ports are recommended, two ports are just fine most of the time for a two-node CDOT, especially in a switchless configuration.  In fact, the latest Data_ONTAP_83_Express_Setup_Guide_for_80xx_Systems (https://www.dropbox.com/s/azuo4rdglm3qg9n/Data_ONTAP_83_Express_Setup_Guide_for_80xx_Systems.pdf?dl=0) shows you how.  

Typical clusters show tens of KB/s on the cluster-interconnect during most operations when LIFs are on the same node as the volumes (and ultimately the aggregates) they are serving.  Cluster ring database updates don’t push that much data.  Even when LIFs get migrated off-node and data is served out indirect, two 10GbE ports in your cluster interconnect should give you enough bandwidth to handle this.  If you intend to not manage LIF/volume residency at all and let LIFs and volumes fall on whatever node they may, then additional cluster interconnect links will buy you some protection.  Otherwise, going four links for cluster interconnect will pay off the most if you do frequent volume relocates across nodes, or once your cluster grows with additional node pairs.

On Apr 9, 2015, at 5:46 AM, Sayla, Mustafa <MSAYLA@mesirowfinancial.com> wrote:

Thank you for your response. I verified we have the X6599A SFP. Just to clarify what I meant to ask is that can we have two ports go to one pair of Catalyst 6500 switch and 2 ports go to another pair of Catalyst 6500 switch and tie all four ports into one interface group so I have 40GB and also have switch redundancy. If not possible what is the best configuration I can have to have 40GB throughput.

 

Mustafa Sayla

 

From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of tmac
Sent: Thursday, April 09, 2015 7:52 AM
To: Clark, André M.
Cc: toasters@teaparty.net
Subject: Re: FAS8040 network configuration in CDOT

 

Yeah, keep looking back at the FAS8020 for some reason...only has the four ports.

 

Make sure you have the X6599A (the 10-GIG SFP) and not the X6596(FC SFP).

For each controller:

a0a should consist of 

e0e/e0g which should go to the same switch, if possible, different blades on the same switch and

e0f/e0h which should go to the same switch, if possible, different blades on the same switch

--> if you cannot split across blades, at least split across ASICs on the same blade.

--tmac

 

Tim McCarthy

Principal Consultant

 

 

 

On Thu, Apr 9, 2015 at 8:24 AM, Clark, André M. <Andre.Clark@redeight.com> wrote:

Each 8040/60/80 controller has 4 10GbE ports (e0a-e0d) and 4 UTA2 ports (e0e-e0h).  Ports e0a-e0d, in a cDOT configuration, are used for the private cluster network while ports e0e-e0h are used for data communications.  No extra 10G card is needed at this time.

 

Regards,

 

André M. Clark | Sr. Consulting Architect | 917.388.8236
Tell me I will forget... Show me I may remember... Involve me I WILL UNDERSTAND!!!
Start by doing what's necessary, then what's possible, and suddenly you are doing the impossible!!!

Red8, An Insight Investments Company

www.redeight.com

 

From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of tmac
Sent: Thursday, 9 April, 2015 06:22
To: Francis Kim
Cc: toasters@teaparty.net
Subject: Re: FAS8040 network configuration in CDOT

 

No that won't work.

If you are configuring cDOT, at least two ports per controller are reserved for the cluster network (e0a & e0c)

This leaves e0b and e0d available for normal network access

 

The recommended configuration is to use all four ports for the cluster network (e0a, e0b, e0c & e0d).

In this case, if you need/want 10gig network connections you would need an extra 10gig card

 

AS indicated earlier in this thread, from each controller:

a0a should be a multimode_lacp VIF and contain:

e0b -> switch 1

e0d -> switch 2

 

Then you can create your VLANs: a0a-123, a0a-456 

 

Place your LIFs on the tagged VLANs

 

Create a failover group for node-01:a0a-123 and node-02:a0a-123

Create a failover group for node-01:a0a-456 and node-02:a0a-456

 

Assign the failover group to the appropriate LIF

 

PS: DO NOT FORGET to disable spanning tree on the switch ports.

--> spanning tree protocol should only be enabled on ports connected to other switches.

--> it is not needed on ports connected to en hosts

--> in a cluster failover event it could take an extra 45 seconds for a port to become available.(not good)

--> remember to use the right version of the command. if it is a trunk, you usually need that on the line

 

--tmac

 

Tim McCarthy

Principal Consultant

 

 

 

On Wed, Apr 8, 2015 at 11:36 PM, Francis Kim <fkim@berkcom.com> wrote:

You can also tie the four links into a single interface group and carve it up into multiple VLANs.

 

What ports are you going to use for the cluster interconnect?  I assume you’re ordering a switchless cluster of two nodes?

 

 

Francis Kim | Engineer

510-644-1599 x334 | fkim@berkcom.com

 

BerkCom | www.berkcom.com

NetApp | Cisco | Supermicro | Brocade | VMware

 

On Apr 8, 2015, at 3:51 PM, Sayla, Mustafa <MSAYLA@mesirowfinancial.com> wrote:

 

We are looking to buy new FAS8040 and configure it with CDOT. I am looking for network recommendation with Cisco 6509 VSS. 8040 has four 10G ports per controller and I want to use all four ports to create an interface group with two VLAN, using VLAN tagging. I want 2 ports to go to one VSS switch and 2 ports to go to other VSS switch so I have switch redundancy also. I am not sure if it is doable since I am going to two different switches. Any recommendation is appreciated.

 

Mustafa

Visit us on the Web at mesirowfinancial.com 

This communication may contain privileged and/or confidential information. It is intended solely for the use of the addressee. If you are not the intended recipient, you are strictly prohibited from disclosing, copying, distributing or using any of this information. If you received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. Confidential, proprietary or time-sensitive communications should not be transmitted via the Internet, as there can be no assurance of actual or timely delivery, receipt and/or confidentiality. This is not an offer, or solicitation of any offer to buy or sell any security, investment or other product._______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

 

_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

 

 

Visit us on the Web at mesirowfinancial.com 

This communication may contain privileged and/or confidential information. It is intended solely for the use of the addressee. If you are not the intended recipient, you are strictly prohibited from disclosing, copying, distributing or using any of this information. If you received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. Confidential, proprietary or time-sensitive communications should not be transmitted via the Internet, as there can be no assurance of actual or timely delivery, receipt and/or confidentiality. This is not an offer, or solicitation of any offer to buy or sell any security, investment or other product._______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters