Are your NFS clients also connected to both switches using two IPs
on the same subnet? Perhaps the NFS clients are having trouble with
the routing. In this case two subnets will probably fix it.
No, the nfs clients only have one nic/ip on the subnet.
Are the two switches interconnected? In particular are ARP requests,
etc., visible to ports on both switches? If not, then I think
you definitely need two different subnets, one for each switch.
Yes, the two switches are interconnected. I'm not a networking guy so I'm not sure about the ARP requests but I believe the switches are connected to each other via two 1gig ports and can communicate through each other if needed.