On Tue, Aug 20, 2002 at 01:56:08PM -0700, Jeff Bryer wrote:
Not urban legend for us. We've experienced this just last week.
We have a new F810 with a dedicated GigE connection to an E880 (Sol8), intending to use the F810 as our mail spool device.
NFS mounting across UDP we can write to the NetApp at 60MB/s. Going from the NetApp to the SUN it goes at 25-30MB/s once or twice, and then slows right down to 1-2MB/s.
Mounting NFS across TCP, we see 50MB/s to the NetApp, and 25MB/s going back from the NetApp to the SUN.
With our production F740 mounting NFS/UDP across 100mb to our SUNs (Sol 2.6 and 8) we've had no performance problems.
After reading through responses, I get a feel that
1) the alleged Solaris bug only appears with Gb connection involved ( it wasn't clear in the original messages ). I only have 100Mb Suns with 1 exception.
2) the problem only occurs with NFS reads - NFS writes are fine ( only Jeff Bryer said that explicitely )
There're some things I still don't understand. Jeff says that his F810 is connected directly with a Sun - no switch in between I presume. If it's the case, then it pretty much eliminates network switches as a source of problem.
I have a V880 server with a Gb interface, and it talks to a NetApp's Gb interface through Extreme Network's Summit48 switch. Preliminary testing didn't indicate any NFS read problem, and I'm still deciding on the course of further testing.
I also found Sun bug #4434256, which talks about slow reads over UDP.
I'll quote it here:
\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\ When Solaris8 read a remote file using NFSv3/UDP, Solaris8 sometimes wait 0.8 sec before sending the READ request to the server.
Following is a network configuration of customer's site.
FirstEther Gigabit E250 ------------- SSR2000 ------------ NetApp (Sol 8)
2332 0.00005 NetApp -> Sol8 UDP IP fragment ID=17850 Offset=31080 MF=1 2333 0.00006 NetApp -> Sol8 UDP IP fragment ID=17850 Offset=32560 MF=0 2334 0.84602 Sol8 -> NetApp NFS C READ3 FH=5AA1 at 7503872 for 32768 ~~~~~~~ 2335 1.73994 Sol8 -> NetApp NFS C READ3 FH=5AA1 at 7503872 for 32768 (retransmit) 2336 0.00051 NetApp -> Sol8 UDP IP fragment ID=18106 Offset=0 MF=1 2337 0.00010 NetApp -> Sol8 UDP IP fragment ID=18106 Offset=1480 MF=1
Since there is a speed difference between client and server, the switch has caused buffer overflow. So It caused retransmission of READ packet from Solaris 8. In order to lessen transmission of read request Packet, the user has set nfs3_nra parameter to 1.
However,It is not necessary to wait to send a FIRST read request packet even in such a environment. \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\ Can any confirm this? The workaround of course is to use NFS over TCP.
Igor