Sun apparently changed something in the way multiple IP's assigned to a single interface behaves in Solaris 2.6. As soon as I turn on hme0:1 my Netapp mounted read/write file system starts behaving oddly. And, if I reboot the machine, then I get a permission denied message when I try to mount.
I noticed that the Netapp's arp cache picked up the second ip as soon as I turned it on on the Solaris 2.6 box. On my Solaris 2.5.1 box that sits right next to it, its second IP never shows up in the Netapp arp table. And, the Solaris 2.5.1 box happily mounts things.
I tried putting both of the Solaris 2.6 box's ip's in the Netapp /etc/exports file, but I still get a "permission denied" message when I try to mount. Turning off the interface and rebooting the Solaris 2.6 machine fixes things. I can mount the file system again.
I really do need multiple IP addresses on these Solaris 2.6 machines though. Is there some sort of workaround that is a little less drastic than reverting to Solaris 2.5.1?
(The Filer is a 630 running 4.1c if it makes a difference)
-Rasmus
Ugggggh. This is biting me too.
See:
http://sunsolve.sun.com/private-cgi/us/doc2html?bug/kernel/tcp-ip/4081315+87... 6654722
None of their proposed workarounds work here. And I don't know the current status of any more elegant workarounds or fixes.
See comp.unix.solaris for more details (especially posts from Casper Dik).
-- Dave Pascoe | mailto:dave@mathworks.com | Voice: 508.647.7362 KM3T | http://www.mathworks.com | FAX: 508.647.7002 PGP fingerprint: 53 AD 71 88 2F AA 45 AC D0 2E 68 91 71 77 39 AF
For those who don't have Sunsolve:
Bug Id: 4081315 Category: kernel Subcategory: tcp-ip State: closed Synopsis: Logical interface addresses are used as IP source addresses. Description:
A number of people with webservers and such with multiple interface addresses for each physical interface found on upgrading to Solaris 2.6 that the behaviour of these interfaces has changed.
In Solaris 2.5.1 and before, the local address used for outgoing connections would always be the physical (primary, logical unit :0) address. In Solaris 2.6 this behavious has changed; the kernel now alternates between addresses.
If the "group ifs" are disabled, the kernel will pick one address at random and use that.
For a number of sites this is a big problem:
- filtering on source addresses (e.g., use of remote sunsolve) - NFS exports (now potentially thousands of address aliases must be added to the export list of servers serving these clients)
Using interface groups for locgical interfaces seems to serve no useful purpose. Not being able to turn this behaviour of is even worse.
BTW, the only change you get with "ndd -set ip_enable_group_ifs 0" is that the addresses are no longer rotyated; instead a random one is picked. Work around:
Integrated in releases: Duplicate of: 4077132 Patch id: See also: Summary:
Here's more:
Bug Id: 4077132 Category: kernel Subcategory: tcp-ip State: evaluated Synopsis: ip_enable_group_ifs=0 doesn't pick default hostname. Description:
cu has multiple virtual interfaces designated for le0. This worked fine on 2.5.1. Cu upgraded to 2.6 and now finds that a new feature of 2.6 (interface groups) has begun selecting interfaces at random. Solaris 2.6, as it is shipped is supposed to operater this way to make load handling more efficient.
However, by setting the ndd option ip_enable_group_ifs to 0, it disables the interface groups functionality and allows 2.6 to operate like a 2.5.1 or previous OS release.
The problem is that when the interface groups are disabled, the hostname selected to handle network calls is the last hostname.le0:[0-9] file found. I tested this on my workstation and found that no matter what the last hostname.le0:? file was, the last file found would become the interface the system always used for network calls.
This is a bad thing for customers who use host authentication and have an ever-changing network...like webservers.
When interface groups are disabled, the loghost (hostname.le0) should be selected.
Work around:
Have the hostname.le0 host name be selected when the interface groups are disabled.
casper.dik@Holland 1997-10-02
[[ DELETED WRKAROUND ]] <--WHY?????????????????????????
This workaround works for EVERY SINGLE CASE I have seen to date.
Do not delete this workaround. Even if it only works part of the time, it is still better than nothing.
The workaround, as stated previously is to (prior to deletion):
1. cp hostname.<interface> (e.g. hostname.le0) to hostname.<interface>:99.
NOTE: You must use 99 because there will never, ever be a file listed in higher than that (unless there are over 1000 virtual interfaces, then 999 would need to be used).
2. verify that hostname.<interface>:99 matches the contents of /etc/nodename.
3. change the contents of hostname.<interface> to some other hostname.
Please do not delete this workaround. Even if it doesn't work all of the time, it is better than nothing.
It didn't work around the problem all of the time. <- see above
If deleting routes doens't work around theproblem that can only indicate that a routing problem has been introduced in 2.6, I can't quite put a finger on it yet.
If routing worked properly, the following should work around the problem:
ifconfig le0:X metric 100
or route delete <interface route>
However, since neither workaround works we do have a problem.
steve.hodnett@East 1997-10-23 Startup script workaround can be added to rc2.d
/etc/rc2.d/S70fixroutes #!/bin/sh # echo "Turning off ifgrps" /usr/sbin/ndd -set /dev/ip ip_enable_group_ifs 0 # echo "Removing Duplicate Virtual Interfaces Network routes" /usr/sbin/route delete net <ip#ofle0:1> <network#> /usr/sbin/route delete net <ip#ofle0:2> <network#> .... /usr/sbin/route delete net <ip#ofle0:xx> <network#>
casper.dik@Holland 1997-10-28
I deleted *MY* workaround because it didn't work.
The route delete workaround also doesn't work. Integrated in releases: Duplicate of: Patch id: See also: 4081315 Summary:
BTW, the only change you get with "ndd -set ip_enable_group_ifs 0" is that the addresses are no longer rotyated; instead a random one is picked. Work around:
This looks like it might do the trick. I haven't tested it yet, but, theoretically if you do the above ndd and then put each IP into the NetApp's /etc/exports file, it should work.
Would certainly be nice if Sun would let you turn that "bug" off and also let you specify which of the virtual ip's should act as the source IP.
-Rasmus
-
Dave Pascoe -
rasmus＠lerdorf.on.ca