Thank you,
That really helped (boosted TPS from 38 to 83 :). Is this option documented or not? I can't find it in the manual...
P.S. Still a big problem with NFS2 performance. We have (besides of Solaris) lots of IRIX boxes, and (surprise) if you use automount on IRIX all mounts are NFS2 only...
-----Original Message----- From: Keith Brown [mailto:keith@netapp.com] Sent: Wednesday, March 22, 2000 4:23 PM To: Rainchik, Aleksandr (MED, Non GE) Subject: Re: Solaris 7 & NFS2/NFS3/TCP/UDP....
b) There is no way to use w/rsize=32768 with NetApp with NFS3 over UDP. Why? I can do it when mounting from Solaris.
Did you set the nfs.udp.xfersize set to 32768? Like so....
filer> options nfs.udp.xfersize 32768
Keith
That really helped (boosted TPS from 38 to 83 :). Is this option documented or not? I can't find it in the manual...
That option has been around for some time, although it has never seemed to find its way onto the na_options(1) man page, for reasons I'm afraid I don't know. It was just bumped to having a 32K default in the latest 5.3.5 releases. I *think* we really should be documenting that option though. Sorry. It's one of the most celebrated "secret" options we have! :-)
P.S. Still a big problem with NFS2 performance. We have (besides of Solaris) lots of IRIX boxes, and (surprise) if you use automount on IRIX all mounts are NFS2 only...
Really. So IRIX doesn't let you put mount options in its /etc/auto_master entries? That's a little unhip.
The usual drill for NFS mounting when no options are specified, as defined by Sun/Solaris (if you accept Sun/Solaris as a definer here?), is to mount using the highest revision of the NFS protocol available on both the client and the server, and the first connection oriented protocol available (usually TCP), falling back to the first connectionless protocol (usually UDP). So if the IRIX folk were paying any attention to the Solaris folk when they put their implementation together, as long as you have:
filer> options nfs.v3.enable on filer> options nfs.tcp.enable off
set on the filer, you would be in with a fighting chance of getting your 32K/NFSv3/UDP aspirations satisfied, even without specifying any mount options (32K xfer sizes are the default for NFSv3).
Keith
I just looked at our main filer and realized we are using 8k NFS packets. With almost exclusively solaris 2.6 and 2.7 clients. I bet we would see a significant speed increase if we upped it, particularly with the large regressions being done on this Netapp. But, can I just log in and change the value? Will that affect existing mounts, and at what point would the new blocksize take effect? Or, can I only set it on reboot -- but even then the clients would have to remount the filesystem right? Someone help me out on the proper way to do this, and let me know if doing this causes harm in any way. Looking at nfsstat most of our operations are indeed nfs3.
Justin Acklin
-----Original Message----- From: Keith Brown [mailto:keith@netapp.com] Sent: Wednesday, March 22, 2000 4:23 PM To: Rainchik, Aleksandr (MED, Non GE) Subject: Re: Solaris 7 & NFS2/NFS3/TCP/UDP....
b) There is no way to use w/rsize=32768 with NetApp with NFS3 over UDP. Why? I can do it when mounting from Solaris.
Did you set the nfs.udp.xfersize set to 32768? Like so....
filer> options nfs.udp.xfersize 32768Keith
I bet we would see a significant speed increase if we upped it, particularly with the large regressions being done on this Netapp.
The chances are probably good, although mileages can vary.
But, can I just log in and change the [nfs.udp.xfersize] value?
Yes you can. There is no need to reboot the filer or anything.
Will that affect existing mounts, and at what point would the new blocksize take effect?
No, it won't effect existing mounts. You will have to pick your moment to unmount and remount your file systems after setting this parameter, in order for the new configuration to take effect for your various clients.
Someone help me out on the proper way to do this, and let me know if doing this causes harm in any way.
Well, set the option, and try one client first (umount and remount its file systems with "-o vers=3,proto=udp,rsize=32768,wsize=32768" etc..., so you're certain of what you're getting). If you find it helps performance, then go to town. :-)
Keith
On Wed, Mar 22, 2000 at 04:39:14PM -0800, Keith Brown wrote:
I bet we would see a significant speed increase if we upped it, particularly with the large regressions being done on this Netapp.
The chances are probably good, although mileages can vary.
But, can I just log in and change the [nfs.udp.xfersize] value?
Yes you can. There is no need to reboot the filer or anything.
Will that affect existing mounts, and at what point would the new blocksize take effect?
No, it won't effect existing mounts. You will have to pick your moment to unmount and remount your file systems after setting this parameter, in order for the new configuration to take effect for your various clients.
Someone help me out on the proper way to do this, and let me know if doing this causes harm in any way.
Well, set the option, and try one client first (umount and remount its file systems with "-o vers=3,proto=udp,rsize=32768,wsize=32768" etc..., so you're certain of what you're getting). If you find it helps performance, then go to town. :-)
Even with the settings as above, I'm getting a worse by about factor of 2-3 performance on Solaris client, compared to Linux clients. Can any1 explain that? I also used Postmark to compare performance.
Thanks
Igor
Well, set the option, and try one client first (umount and remount its file systems with "-o vers=3,proto=udp,rsize=32768,wsize=32768" etc..., so you're certain of what you're getting). If you find it helps performance, then go to town. :-)
Even with the settings as above, I'm getting a worse by about factor of 2-3 performance on Solaris client, compared to Linux clients. Can any1 explain that? I also used Postmark to compare performance.
Yeah. I've seen this as well on 2.7/sparc. Wall clock times were outrageously high for both read and write operations, however when I tried the same on 2.6 or Linux, performance was perfectly acceptable.
I did see a problem with excessive reassembly timeouts, which was peculiar, as I definately didn't miss any packets (though it's possible that something munged a checksum - I didn't have that much time to look at it :) ). Saw it on two 760's, both attached via the intel gbit cards to a catalyst 6509, with the hosts on 100fdx (same vlan, no routing). After looking at this though, they still didn't justify the times we were seeing.
It definately does have something to do with large-packets, so the knee-jerk response from netapp for performance to set nfs.udp.xfersize to 32k was detrimental. By tuning it down to 8k, everything acted normal again.
..kg..
-
Igor Schein -
Justin -
Keith Brown -
kevin graham -
Rainchik, Aleksandr (MED, Non GE)