They are 10 gig cards, they are only in a VIF with themselves (no other cards). It’s a 3070 with 2 copper 100/1000 2 FC gig and 2 FC 10g connections. The 2 1000 are on our production network as lan0 VIF for CIFS/NFS, the fibre gig is not used and the two 10g FC are VIF tgif on the storage only network (no routing etc, private for storage devices and the hosts using them only).

They are plugged into stacked 3750e boxes, I tried putting them on the same switch just to make sure it was not a stacking thing with no real change.

The oddness is that occasionally a single packet will get through, and if I bring the interfaces up and down it may change who can get to what.

From: Webster, Stetson [mailto:Stetson.Webster@netapp.com]
Sent: Thursday, May 15, 2008 2:13 PM
To: Page, Jeremy; toasters@mathworks.com
Subject: RE: VIF weirdness

Are these 10g TOE cards? Are they in a VIF with other 'non-TOE' cards?

Stetson M. Webster
Onsite Professional Services Engineer
PS - North Amer. - East

NetApp
919.250.0052 Mobile
Stetson.Webster@netapp.com
www.netapp.com

From: Page, Jeremy [mailto:jeremy.page@gilbarco.com]
Sent: Thursday, May 15, 2008 1:32 PM
To: toasters@mathworks.com
Subject: VIF weirdness

I am having a great deal of difficulty getting my filer’s 10g interfaces connected to a pair of 3750E switches I am trying to use for my storage network’s backbone. I can inconsistently ping other objects on the network (there really is nothing there yet, we are just getting started, but I do have a host and an RLM card just for troubleshooting purposes), but they cannot ping each other. >From a remote host (172.1.2.5) I can ping everything but array1. If I bring down either of the two stacked switches (which brings down one port of each VIF member pairs) everything works.

array01> ping 172.1.0.1

172.1.0.1 is alive

array01> ping 172.1.0.2

array01> ping 172.1.0.4

ping: wrote 172.1.0.4 64 chars, error=Host is down

array01> Thu May 15 13:28:33 EDT [gvr-array01: nis_worker_0:info]: Local NIS group update successful.

array01> ping 172.1.2.5

172.1.2.5 is alive

array02> ping 172.1.0.1

array02> ping 172.1.0.4

172.1.0.4 is alive

array02> ping 172.1.0.4

172.1.0.4 is alive

array02> ping 172.1.0.250

172.1.0.250 is alive

array02> ping 172.1.2.5

no answer from 172.1.2.5

array01> ifconfig -a

e0a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

ether 02:a0:98:08:22:b7 (auto-1000t-fd-up) flowcontrol full

trunked lan0

e0b: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

ether 00:a0:98:08:22:b6 (auto-unknown-cfg_down) flowcontrol full

e0c: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

ether 02:a0:98:08:22:b7 (auto-1000t-fd-up) flowcontrol full

trunked lan0

e0d: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

ether 00:a0:98:08:22:b4 (auto-unknown-cfg_down) flowcontrol full

e2a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

ether 02:a0:98:08:22:b6 (auto-10g_sr-fd-up) flowcontrol full

trunked tgif

e2b: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

ether 02:a0:98:08:22:b6 (auto-10g_sr-fd-up) flowcontrol full

trunked tgif

lo: flags=1948049<UP,LOOPBACK,RUNNING,MULTICAST,TCPCKSUM> mtu 8160

inet 127.0.0.1 netmask 0xff000000 broadcast 127.0.0.1

ether 00:00:00:00:00:00 (VIA Provider)

lan0: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

inet 10.28.17.213 netmask 0xffffff00 broadcast 10.28.17.255

partner 10.28.17.214 (not in use)

ether 02:a0:98:08:22:b7 (Enabled virtual interface)

tgif: flags=4948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM,NOWINS> mtu 1500

inet 172.1.0.1 netmask 0xffff0000 broadcast 172.1.255.255

partner 172.1.0.2 (not in use)

ether 02:a0:98:08:22:b6 (Enabled virtual interface)

nfo enabled

array02>ifconfig –a

e0a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

ether 02:a0:98:0c:1a:37 (auto-1000t-fd-up) flowcontrol full

trunked lan0

e0b: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

ether 00:a0:98:0c:1a:36 (auto-unknown-cfg_down) flowcontrol full

e0c: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

ether 02:a0:98:0c:1a:37 (auto-1000t-fd-up) flowcontrol full

trunked lan0

e0d: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

ether 00:a0:98:0c:1a:34 (auto-unknown-cfg_down) flowcontrol full

e2a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

ether 02:a0:98:0c:1a:36 (auto-10g_sr-fd-up) flowcontrol full

trunked tgif

e2b: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

ether 02:a0:98:0c:1a:36 (auto-10g_sr-fd-up) flowcontrol full

trunked tgif

lo: flags=1948049<UP,LOOPBACK,RUNNING,MULTICAST,TCPCKSUM> mtu 8160

inet 127.0.0.1 netmask 0xff000000 broadcast 127.0.0.1

ether 00:00:00:00:00:00 (VIA Provider)

lan0: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

inet 10.28.17.214 netmask 0xffffff00 broadcast 10.28.17.255

partner 10.28.17.213 (not in use)

ether 02:a0:98:0c:1a:37 (Enabled virtual interface)

tgif: flags=4948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM,NOWINS> mtu 1500

inet 172.1.0.2 netmask 0xffff0000 broadcast 172.1.255.255

partner 172.1.0.1 (not in use)

ether 02:a0:98:0c:1a:36 (Enabled virtual interface)

nfo enabled

This message (including any attachments) contains confidential
and/or proprietary information intended only for the addressee.
Any unauthorized disclosure, copying, distribution or reliance on
the contents of this information is strictly prohibited and may
constitute a violation of law. If you are not the intended
recipient, please notify the sender immediately by responding to
this e-mail, and delete the message from your system. If you
have any questions about this e-mail please notify the sender
immediately.