While this is true for NFS, I've seen where lockd and some other UDP based protocols would respond from another interface on the same subnet (quotad and SNMP is a good example) . I can currently reproduce with SNMP.
Example
192.168.1.1 - VIF 1 IP
192.168.1.2 VIF 2 IP
mount command against VIF 1 will see a response from VIF2 IP for some of protocols like mount, snmp, The actual *NFS* traffic (port 2049) will always come from VIF1. I don't necessarily see an issue with this, as the RPC mechanisms are higher level than the IP stack, but some OS's might not like this asymmetry.
I can currently reproduce this with SNMP on 7.3.1.1P8.
I pull VIF 1, but the return traffic was coming from VIF2. the host based firewall did not like this, so I just configured the polling box to use VIF2 and it was happy. I would speculate that it'll happen even if the two interfaces aren't vifs, but I haven't tested.
Yes, this is exactly what is happening to me. Trying a nfs mount to vif2 with "proto=tcp" the filers RPC udp mount/port responses come back over vif1 causing about a 30 second hang on mounts on my Suse box and the Solaris boxes never seem to mount it. Though (on Suse) after the export is mounted all traffic goes through the vif2.
Thanks for the RFE, I second it.
I guess putting the vifs on seperate vlans may be the only way to get around this issue...
Note: Disabling the firewall on my linux box didn't seem to help.