I have two clustered pairs of F740's arriving soon, so I will likely be able to discover answers to most of my questions, but I figured I'd do a bit of legwork and check my assumptions while I'm still in the planning stages.
I'll consider just one of the clustered pairs: nfs1 and nfs2. Both will be equipped with two QFE NIC's. They will connect to a pair of switches with an uplink between them (i.e., passing traffic between two hosts on the same subnet, but on different switches). Logically, consider it a single switch. Each Netapp will be visible on two subnets: 10.35.8.0/25 and 10.35.8.128/25. NFS clients in one subnet will not be able to see filer interfaces in the other.
I want to create two single-mode VIF's on each filer, using four interfaces on each filer:
====Active VIF_Subnet1====[e1a] [e2a]----Standby VIF_Subnet1--- ----Standby VIF_Subnet2---[e1b] [e2b]====Active VIF_Subnet2==== ..........Unused..........[e1c] [e2c]..........Unused.......... ..........Unused..........[e1d] [e2d]..........Unused..........
Trunks are created across NIC's. Ports on e1 are cabled to switch 1, and ports on e2 are cabled to switch 2. Since this is a single- mode trunk, the switches shouldn't care, right? During normal operation, switch 1 only sees an active port from e1a, and switch 2 only sees an active port from e2b (since e2a and e1b are effectively "downed" interfaces).
While it would be a bit less confusing to say "all interfaces on e1 are active, and all interfaces on e2 are on standby", I prefer to have an active interface on each NIC at all times. This should lessen the chance of Murphy's Law kicking in on an early Sunday morning when e1 does fail, only to reveal that e2 had also died quietly sometime before then.
I also want those same physical interfaces to assume their partner's VIF configuration in a failover situation (i.e., what the manual calls "shared interface takover"). So this is what I have, based on the 5.3.2 documentation:
nfs1> vif create single VIF_Subnet1 e1a e2a nfs1> vif create single VIF_Subnet2 e2b e1b nfs1> vif favor e1a nfs1> vif favor e2b nfs1> ifconfig VIF_Subnet1 10.35.8.1 netmask 255.255.255.128 partner 10.35.8.2 nfs1> ifconfig VIF_Subnet2 10.35.8.129 netmask 255.255.255.128 partner 10.35.8.130
nfs2> vif create single VIF_Subnet1 e1a e2a nfs2> vif create single VIF_Subnet2 e2b e1b nfs2> vif favor e1a nfs2> vif favor e2b nfs2> ifconfig VIF_Subnet1 10.35.8.2 netmask 255.255.255.128 partner 10.35.8.1 nfs2> ifconfig VIF_Subnet2 10.35.8.130 netmask 255.255.255.128 partner 10.35.8.129
This configuration should be able to withstand failure of a filer NIC port, a filer NIC, an Ethernet cable, a switch port, an entire switch and an entire filer. Certain combinations of double failures are also non-lethal.
Since this is all theoretical (to me, anyway), I have some questions. :)
What characters are allowed in a VIF name?
Is anyone using single-mode VIF's across two different switches like this?
How does the Netapp know which VIF to use with the "vif favor" command? I assume this means a physical interface can only participate in one VIF at a time.
Is it possible to prioritize interfaces in a VIF with more than two ports trunked together?
Can VIF's be shared in a cluster? Can they share ports as above? Special mention is only made about super VIF's and clustering in the 5.3.2 SAG.
Can super VIF's be created from single-mode VIF's, or only multi-mode VIF's? Again, the manual only mentions the latter, but does not prohibit the former.
Can super VIF's themselves be multi-mode, or single-mode only?
I'll post the results of my configuration testing once I have the hardware to play with.
On Sun, 24 Oct 1999, Brian Tao wrote:
nfs1> vif create single VIF_Subnet1 e1a e2a nfs1> vif create single VIF_Subnet2 e2b e1b nfs1> vif favor e1a nfs1> vif favor e2b nfs1> ifconfig VIF_Subnet1 10.35.8.1 netmask 255.255.255.128 partner 10.35.8.2 nfs1> ifconfig VIF_Subnet2 10.35.8.129 netmask 255.255.255.128 partner 10.35.8.130
I think it should be:
nfs1> ifconfig VIF_Subnet1 10.35.8.1 netmask 255.255.255.128 partner VIF_Subnet1 nfs1> ifconfig VIF_Subnet2 10.35.8.129 netmask 255.255.255.128 partner VIF_Subnet2
nfs2> vif create single VIF_Subnet1 e1a e2a nfs2> vif create single VIF_Subnet2 e2b e1b nfs2> vif favor e1a nfs2> vif favor e2b nfs2> ifconfig VIF_Subnet1 10.35.8.2 netmask 255.255.255.128 partner 10.35.8.1 nfs2> ifconfig VIF_Subnet2 10.35.8.130 netmask 255.255.255.128 partner 10.35.8.129
and:
nfs2> ifconfig VIF_Subnet1 10.35.8.2 netmask 255.255.255.128 partner VIF_Subnet1 nfs2> ifconfig VIF_Subnet2 10.35.8.130 netmask 255.255.255.128 partner VIF_Subnet2
This is because I think that you cannot use IP addreses with trunk interfaces. At least you couldn't do it last time I set up "multi" trunking on a new rev of DOT.
What characters are allowed in a VIF name?
I have no idea, but I'd skip underlines for legacy purposes. NACs try to resolve the NIC name on takeover i.e. they will try to look up VIF_Subnet2.your.domain. Underlines are not allowed in DNS as I remember.
Keep me posted on the results of your experiments.
Tom
On Mon, 25 Oct 1999 tkaczma@gryf.net wrote:
I think it should be:
nfs1> ifconfig VIF_Subnet1 10.35.8.1 netmask 255.255.255.128 partner VIF_Subnet1 nfs1> ifconfig VIF_Subnet2 10.35.8.129 netmask 255.255.255.128 partner VIF_Subnet2
and:
nfs2> ifconfig VIF_Subnet1 10.35.8.2 netmask 255.255.255.128 partner VIF_Subnet1 nfs2> ifconfig VIF_Subnet2 10.35.8.130 netmask 255.255.255.128 partner VIF_Subnet2
Right, and I think Sam's followup post explains how this works. If nfs1 fails, nfs2 will see that the partner trunk for the first interface is called "VIF_Subnet1", and will look up the corresponding ifconfig statement in nfs1's /etc/rc to map it to an IP address. To use a less ambiguous example:
nfs1> ifconfig Toronto 10.35.8.1 netmask 255.255.255.0 partner Ottawa nfs1> ifconfig Montreal 10.35.8.129 netmask 255.255.255.0 partner Vancouver nfs2> ifconfig Ottawa 10.35.8.2 netmask 255.255.255.0 partner Toronto nfs2> ifconfig Vancouver 10.35.8.130 netmask 255.255.255.0 partner Montreal
This is because I think that you cannot use IP addreses with trunk interfaces.
That doesn't seem to be the case, since I can ifconfig an IP address to a trunk group. The benefit, as I see it, is that it forces the specification of the IP address to a single location. There is less chance that you'll make an IP typo in the "partner" statement. Also, if you change the IP address on the active filer's /etc/rc, the partner filer automatically knows about it, without requiring additional changes to its /etc/rc.
I have no idea, but I'd skip underlines for legacy purposes. NACs try to resolve the NIC name on takeover i.e. they will try to look up VIF_Subnet2.your.domain.
Oh? I don't think that is mentioned in the manual. VIF names need to have corresponding IP addresses in the filer's /etc/hosts then?
On Tue, 26 Oct 1999, Brian Tao wrote:
I have no idea, but I'd skip underlines for legacy purposes. NACs try to resolve the NIC name on takeover i.e. they will try to look up VIF_Subnet2.your.domain.
Oh? I don't think that is mentioned in the manual. VIF names
need to have corresponding IP addresses in the filer's /etc/hosts then?
I don't know why they try to resolve the name. I think it's just a mess remaining from the fact that before you could specify an IP address for a partner trunked interface.
Tom