Thus spake Alexei Rodriguez, on Mon, Mar 04, 2002 at 10:38:22AM -0800:
On Mon, 2002-03-04 at 09:41, Jose Celestino wrote:
We are currently experiencing some heavy load on a filer serving as storage to a webmail farm:
Which model filer?
Sorry, I failed to point that out, F760.
The filer volume webmail:
FILER> df Filesystem kbytes used avail capacity Mounted on /vol/webmail/ 406736736 383407488 23329248 94% /vol/webmail/ /vol/webmail/.snapshot 101684180 0 101684180 0% /vol/webmail/.snapshot
Looks like you have the snapshot reserve set to 20% and dont use it. You should recover that space. The more space you give the filer on the volume, the better ;)
Goddamn! You're right, that really passed me by. 76% free now, that should really relieve the CPU a bit :)
The getattr seems way too big and this may point to a bad caching on the frontends. But could this bring the CPU to 100% most of the time? Could this be a wafl issue related with the low available space on the volume?
Are the clients doing v2 or v3 NFS mounts? Old data (from a previous life) suggests that for mail and news applications v2 edges out v3. From the looks of it you have both types of access going on.
Exactly, I have both, client NFS versions differ slightly. I noticed that also. Should I use only v2 or only v3 ?
Any ideas to help optimize the performance in this scenario? Any ideas are welcome.
It looks (from the 22 second snapshot of sysstat) that the writes are when the filer is getting pushed. Some things to look at as possible improvements (some require more work than others):
- what do the raid groups look like? smaller groups may help the writes
FILER3> vol status -v Volume State Status Options webmail online normal nosnap=on, nosnapdir=off, minra=off, no_atime_update=off,raidsize=14, nvfail=off raid group 0: normal raid group 1: normal
- if snapshots are disabled (which makes sese for a mail filer), then recover the snap reserve space.
Done.
- how much read cache do you have in the filer? Max it out.
I have minra=off, are you talking about this?
- it looks like there is a fair ammount of network activity; you might want to enable a vif or upgrade to gige.
Yes, about 40Mbit in peaks. We are considering trunking 2x100Mb or going to Gb.
- what OS do the clients run? some have better nfs performance than others. Perhaps there are newer versions out.
Linux 2.2.19 only. 2.4.18> possibly soon.
You might also be at the breaking point for the filer (but I think you can get a bit more out of it by making some of the changes listed above).
At what rate do you add mailboxes and grow the data on the filer?
No maildirs are being added to this filer. The Maildirs are being created on another filer, but the data grows about 250/300 Mbytes a day.
I also noticed that we had no_atime_update=off, this seems useless, I have turned it to no_atime_update=on.
alexei