If you have multiple network interfaces on your filers, this information may be useful to you. If you're thorough about reading the documentation and clever about checking your results, you may have discovered it for yourself.
While testing our new F760 cluster, we have discovered that if the filer has two interfaces on the same subnet and the default gateway resides on that subnet, the default route in the filer's routing table will choose the interface with the lower number.
In particular, using the "setup" command, we configured an F760 with on-board 100Mbps e0 and GigE e7 on the same subnet. Using "netstat -r", we determined that the default route was using e0, which was not what we wanted--the 100Mbps was intended to be a backup link, not for regular production use:
pow> netstat -r Routing tables
Internet: Destination Gateway Flags Refs Use Interface default 135.90.100.254 UG 1 61 e0 127 localhost UG 1 9 lo localhost localhost UH 1 0 lo 135.90.100/24 link#1 UC 0 0 e0 135.90.100.254 0:50:50:3f:18:1f UHL 1 0 e0 135.90.100.255 link#1 UHL 1 46 e0
After some experimentation, we settled on editing /etc/rc to configure e0 after both configuring e7 and adding the default route; we also turned off routed:
#Auto-generated by setup Wed Dec 20 19:16:53 GMT 2000 hostname pow ifconfig e7 `hostname`-e7 flowcontrol full netmask 255.255.255.0 route add default 135.90.100.254 1 ifconfig e0 `hostname`-e0 mediatype auto netmask 255.255.255.0 #routed on options dns.domainname is.paradyne.com options dns.enable on options nis.domainname is.paradyne.com options nis.enable on savecore exportfs -a nfs on
Now we have the results we wanted; all outbound packets should use the gig link, according to netstat:
pow> netstat -r Routing tables
Internet: Destination Gateway Flags Refs Use Interface default 135.90.100.254 UG 5 2148 e7 127 localhost UG 1 261 lo localhost localhost UH 1 0 lo 135.90.100/24 link#2 UC 0 0 e7 135.90.100.254 0:50:50:3f:18:1f UHL 1 0 e7 135.90.100.255 link#2 UHL 1 1794 e7
Looking at the systems we had in production, we realized that, as a result of this oversight, they had been sending all outbound traffic via their fast ethernet interfaces rather than gig! Aaargh!
--Brian L. Brush Senior Systems Administrator Paradyne Corporation
"Brian L. Brush" wrote:
Looking at the systems we had in production, we realized that, as a result of this oversight, they had been sending all outbound traffic via their fast ethernet interfaces rather than gig! Aaargh!
I want to correct what I wrote there; according to p. 561 of the ONTAP 5.3 System Administrator's Guide,
"The filer always responds to an NFS request by sending its reply on the interface on which the request was received."
Therefore, because all of our NFS calls were using the gig link, the responses were using the same, and the situation wasn't as bad as we first suspected. Even so, the configuration was not what we intended, so I'm glad our testing discovered our mistake.
On another note, Steve Vawter wrote to mention single-mode trunks for automatic failover between links on a single filer; that is, indeed, the subject of my current testing, but initially we simply wanted to have the onboard link on a different address in case of flakiness while configuring the GigE card (which we had encountered on our old F230). In fact, we've had no such problems with the 700 series.
--Brian L. Brush Senior Systems Administrator Paradyne Corporation
-
Brian L. Brush