On Tue, 9 Nov 1999, Aimee Schmidt wrote:

...

Try installing the appropriate Solaris patches and tune NFS/DNLC parameters. This will improve performance by an order of magnitude or so.

Whether/how much these patches help depends a great deal on the specifics of your situation. Nonetheless, these are probably good things. I wish Sun would let us know exactly how these patches implement their fix.

...

Patches: 5.6 sparc - 105720-08 5.6 i386 - 105721-08 5.7 sparc - 106541-06 5.7 i386 - 106542-06

Tunings (for /etc/system): set nfs:nfs_nra = 10 set nfs:nfs3_nra = 10 set nfs:nfs_max_threads = 24 set nfs:nfs3_max_threads = 24 set ncsize=8192 Aimee

Some of these values are very different from what I've found to be effective on high volume mail systems. I'm not at all sure that ncsize needs to be set this low if the above patches actually fix bug "4212925 NFS client unlink operation slow" effectively, but again, since I don't know what the actual fix introduced was, it's hard to tell.

Does netapp have a white paper discussing suggested solaris tuning parameters with some explanation on why the values were picked?

Not that I know of. The "correct" numbers are application, machine, RAM, and implementation specific, so I doubt they could provide something simple that improved everyone's situation.

My department recently put a 740 in production as a mail spool with a mid of solaris 7 and solaris 8(beta) hosts. I got a list of parameters from another netapp engineer through a salesperson but was reluctant to change the value of nfs:nfs_nra to 24 for example.

For mail systems with small mailboxes I would expect that "10" might be higher than optimal. If you message store is one file per message and most messages are small, I would actually expect performance to be marginally worse with nfs_nra = 10 than with a lower value. I'd hesitate to speculate on what would be optimal without knowing the specific characteristics of a particular situation.

The default value for this appears to be 524288!

This would be several Gigabytes in size. In some situations, automatically loading in the whole file might be a good thing. In some situations, especially where the client connection was very slow, it might read the whole file into memory, but be so slow about sending the connection down to, say, a POP client that LRU caused some of this info to be flushed from the buffer cache before it got transmitted, in which case it would have to be read again, which would be a waste.

Since there are other applications that suggest tuning these parameters, and Sun asks us to change them from time to time, it is helpful to know 'why' we set the values.

Certainly, it is important to understand what each of these parameters controls.

If the real answer is "we just iterated through lots of different numbers and these are the ones that provided the best performance mix", thats OK..

In most cases, this is probably the case. Note, though, that if your servers are memory poor the same parameters that work best on a similar machine with memory to spare are likely to be sub-optimal, for example.

FYI, here is the list of things we got from the salesperson:

set npty = 1024 set nfs:nfs_nra = 10 set nfs:nfs3_nra = 10 set nfs:nfs_max_threads = 24 set nfs:nfs3_max_threads = 24

On really busy mail servers with much memory, my opinion is that nfs_max_threads can be set much higher than this. I believe that one thread per concurrent process accessing data over NFS is not unreasonable as a rule of thumb, as long as the box doesn't start thrashing as a consequence.

set maxusers = 256

Depends on your box and how much RAM you use. For E250, E450, E3500, and larger servers, boosting this to 2048 is not unreasonable if you have RAM to burn in order to gain performance.

set ncsize = 30000

If you don't have one of the patches listed above applied, this may be a little too high. With the patches installed, without knowing how the problem was fixed, it's hard to say whether this number is too high or not.

set doingcache=0

I don't know what this does.

It's impossible to come up with a general set of kernel tuning parameters for Solaris clients doing email over NFS without more information. The nature of the message store is important. The load characteristics are important. What client machines are being used and how much RAM do they have? Is it worth consuming more memory, network, and filer I/O resources to reduce POP or IMAP connection latency? Is the system primarily a POP, file based, or IMAP server? There are a large number of variables that precludes a simple answer.

In everyone's case, there's not much to do about it except to understand what the parameters do and to iterate on them until good things happen. In order to understand what they do, I recommend the 2nd edition of Adrian Cockroft's Solaris performance tuning book and the SMCC NFS Server Performance and Tuning Guide from Sun. For the latter, it's a matter of time and effort. However, for those sites managing less than 100,000 mailboxes, the return on lots of careful tuning is probably small enough that it's not worth pursuing.

One note of caution, Cockroft's book covers Solaris tuning up through about Solaris 2.5.1. While the descriptions are still valid, some of the math and parameters have changed. Not all the advice is good advice if you're running Solaris 2.6 or higher. The same would be true for older editions of the Tuning Guide.