Notes aboutof 10GbE TOE NIC limitations : The 10GbE TOE NIC cards have a number of limitations. They include: ◆ Multimode vif limited to two (2) 10GbE TOE NICs ◆ LACP not supported with 10GbE TOE NICs ◆ TOE functionality disabled on 10GbE NIC in vif
Do you need a switchport access vlan ### in the interface lines?
Here is the config I use for vifs on my switch:
interface Port-Channel7 description netapp-vif switchport switchport access vlan 239 switchport mode access no ip address flowcontrol receive on flowcontrol send on spanning-tree portfast ! interface GigabitEthernet7/42 description filer-e9 switchport switchport access vlan 239 switchport mode access no ip address flowcontrol receive on flowcontrol send on spanning-tree portfast channel-group 7 mode active !
In your case, with 10-gig, the only change should be: channel-group 7 mode on (which you are already doing)
Also, make sure your filer is setup correctly (not using LACP for 10Gig)
vif create multi tgif -b mac e2a r3b
On Thu, May 15, 2008 at 3:55 PM, Page, Jeremy jeremy.page@gilbarco.com wrote:
I *think* the problem may be isolated to array1. I've tried setting them to single mode instead of multi (LACP is not supposed to work on a stacked switch).
array01> sysconfig -v 2 slot 2: Dual TOE-10G Ethernet Controller (T320E-XFP) Device Type: CT-31-1 Version Number: T3-SRAM1.1.0-BR1040-20-C0-FW4.6.0-DR03 Serial Number: PT4107009 e2a MAC Address: 00:07:43:05:16:98 (auto-10g_sr-fd-up) e2b MAC Address: 00:07:43:05:16:99 (auto-10g_sr-fd-up)
**switch config (short form)** version 12.2 no aaa new-model switch 1 provision ws-c3750e-24td switch 2 provision ws-c3750e-24td interface TenGigabitEthernet1/0/1 switchport mode access channel-group 1 mode on ! interface TenGigabitEthernet1/0/2 switchport mode access channel-group 2 mode on interface TenGigabitEthernet2/0/1 switchport mode access channel-group 1 mode on ! interface TenGigabitEthernet2/0/2 switchport mode access channel-group 2 mode on ! interface Vlan1 ip address 172.1.0.250 255.255.0.0
-----Original Message----- From: tmac [mailto:tmacmd@gmail.com] Sent: Thursday, May 15, 2008 3:46 PM To: Page, Jeremy Cc: toasters@mathworks.com Subject: Re: VIF weirdness
How about a "sysconfig -v 2" from the filers and a "show running-config " that shows at a minimum: the port config of the 10-gig stuff and the port-channel definitions.
--tmac
On Thu, May 15, 2008 at 2:19 PM, Page, Jeremy jeremy.page@gilbarco.com wrote:
They are 10 gig cards, they are only in a VIF with themselves (no
other
cards). It's a 3070 with 2 copper 100/1000 2 FC gig and 2 FC 10g connections. The 2 1000 are on our production network as lan0 VIF for CIFS/NFS, the fibre gig is not used and the two 10g FC are VIF tgif on
the
storage only network (no routing etc, private for storage devices and
the
hosts using them only).
They are plugged into stacked 3750e boxes, I tried putting them on the
same
switch just to make sure it was not a stacking thing with no real
change.
The oddness is that occasionally a single packet will get through, and
if I
bring the interfaces up and down it may change who can get to what.
From: Webster, Stetson [mailto:Stetson.Webster@netapp.com] Sent: Thursday, May 15, 2008 2:13 PM To: Page, Jeremy; toasters@mathworks.com Subject: RE: VIF weirdness
Are these 10g TOE cards? Are they in a VIF with other 'non-TOE'
cards?
Stetson M. Webster Onsite Professional Services Engineer PS - North Amer. - East
NetApp 919.250.0052 Mobile Stetson.Webster@netapp.com www.netapp.com
From: Page, Jeremy [mailto:jeremy.page@gilbarco.com] Sent: Thursday, May 15, 2008 1:32 PM To: toasters@mathworks.com Subject: VIF weirdness
I am having a great deal of difficulty getting my filer's 10g
interfaces
connected to a pair of 3750E switches I am trying to use for my
storage
network's backbone. I can inconsistently ping other objects on the
network
(there really is nothing there yet, we are just getting started, but
I do
have a host and an RLM card just for troubleshooting purposes), but
they
cannot ping each other. >From a remote host (172.1.2.5) I can ping everything but array1. If I bring down either of the two stacked
switches
(which brings down one port of each VIF member pairs) everything
works.
array01> ping 172.1.0.1
172.1.0.1 is alive
array01> ping 172.1.0.2
array01> ping 172.1.0.4
ping: wrote 172.1.0.4 64 chars, error=Host is down
ping: wrote 172.1.0.4 64 chars, error=Host is down
array01> Thu May 15 13:28:33 EDT [gvr-array01: nis_worker_0:info]:
Local NIS
group update successful.
array01> ping 172.1.2.5
172.1.2.5 is alive
array02> ping 172.1.0.1
array02> ping 172.1.0.4
172.1.0.4 is alive
array02> ping 172.1.0.4
172.1.0.4 is alive
array02> ping 172.1.0.250
172.1.0.250 is alive
array02> ping 172.1.2.5
no answer from 172.1.2.5
array01> ifconfig -a
e0a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
ether 02:a0:98:08:22:b7 (auto-1000t-fd-up) flowcontrol full trunked lan0e0b: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
ether 00:a0:98:08:22:b6 (auto-unknown-cfg_down) flowcontrolfull
e0c: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
ether 02:a0:98:08:22:b7 (auto-1000t-fd-up) flowcontrol full trunked lan0e0d: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
ether 00:a0:98:08:22:b4 (auto-unknown-cfg_down) flowcontrolfull
e2a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
ether 02:a0:98:08:22:b6 (auto-10g_sr-fd-up) flowcontrol full trunked tgife2b: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
ether 02:a0:98:08:22:b6 (auto-10g_sr-fd-up) flowcontrol full trunked tgiflo: flags=1948049<UP,LOOPBACK,RUNNING,MULTICAST,TCPCKSUM> mtu 8160
inet 127.0.0.1 netmask 0xff000000 broadcast 127.0.0.1 ether 00:00:00:00:00:00 (VIA Provider)lan0: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
inet 10.28.17.213 netmask 0xffffff00 broadcast 10.28.17.255 partner 10.28.17.214 (not in use) ether 02:a0:98:08:22:b7 (Enabled virtual interface)tgif: flags=4948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM,NOWINS>
mtu 1500
inet 172.1.0.1 netmask 0xffff0000 broadcast 172.1.255.255 partner 172.1.0.2 (not in use) ether 02:a0:98:08:22:b6 (Enabled virtual interface) nfo enabledarray02>ifconfig -a
e0a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
ether 02:a0:98:0c:1a:37 (auto-1000t-fd-up) flowcontrol full trunked lan0e0b: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
ether 00:a0:98:0c:1a:36 (auto-unknown-cfg_down) flowcontrolfull
e0c: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
ether 02:a0:98:0c:1a:37 (auto-1000t-fd-up) flowcontrol full trunked lan0e0d: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
ether 00:a0:98:0c:1a:34 (auto-unknown-cfg_down) flowcontrolfull
e2a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
ether 02:a0:98:0c:1a:36 (auto-10g_sr-fd-up) flowcontrol full trunked tgife2b: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
ether 02:a0:98:0c:1a:36 (auto-10g_sr-fd-up) flowcontrol full trunked tgiflo: flags=1948049<UP,LOOPBACK,RUNNING,MULTICAST,TCPCKSUM> mtu 8160
inet 127.0.0.1 netmask 0xff000000 broadcast 127.0.0.1 ether 00:00:00:00:00:00 (VIA Provider)lan0: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
inet 10.28.17.214 netmask 0xffffff00 broadcast 10.28.17.255 partner 10.28.17.213 (not in use) ether 02:a0:98:0c:1a:37 (Enabled virtual interface)tgif: flags=4948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM,NOWINS>
mtu 1500
inet 172.1.0.2 netmask 0xffff0000 broadcast 172.1.255.255 partner 172.1.0.1 (not in use) ether 02:a0:98:0c:1a:36 (Enabled virtual interface) nfo enabledThis message (including any attachments) contains confidential and/or proprietary information intended only for the addressee. Any unauthorized disclosure, copying, distribution or reliance on the contents of this information is strictly prohibited and may constitute a violation of law. If you are not the intended recipient, please notify the sender immediately by responding to this e-mail, and delete the message from your system. If you have any questions about this e-mail please notify the sender immediately.
This message (including any attachments) contains confidential and/or proprietary information intended only for the addressee. Any unauthorized disclosure, copying, distribution or reliance on the contents of this information is strictly prohibited and may constitute a violation of law. If you are not the intended recipient, please notify the sender immediately by responding to this e-mail, and delete the message from your system. If you have any questions about this e-mail please notify the sender immediately.
-- --tmac
RedHat Certified Engineer #804006984323821 (RHEL4) RedHat Certified Engineer #805007643429572 (RHEL5)
Principal Consultant
This message (including any attachments) contains confidential and/or proprietary information intended only for the addressee. Any unauthorized disclosure, copying, distribution or reliance on the contents of this information is strictly prohibited and may constitute a violation of law. If you are not the intended recipient, please notify the sender immediately by responding to this e-mail, and delete the message from your system. If you have any questions about this e-mail please notify the sender immediately.