----- Original Message ----- From: "Todd C. Merrill" tmerrill@mathworks.com To: toasters@mathworks.com Sent: Tuesday, April 04, 2000 7:12 AM Subject: improving filer performance
I am trying to (what else?) improve the performance of the filers that I have. Here are the configs and the environment in which they run. My specific questions follow:
I have two F740's with 14x 18GB FC-AL disks and a trunked 4x 100 Mb/s ethernet card. These filers serve software development sandboxes with a few million files (median file size about 4 kB), not more than a few thousand in the largest directories. These filers are used for both CIFS and NFS client builds; NFS traffic is limited to UDP only (`options nfs.tcp.enable off`) to keep the network overhead down on that protocol. Mix is about 60-75% NFS; the rest, CIFS. Running ONTAP 5.3.4R3P2. CPU load is more often than not above 50%, often 75-90% or more; usually 3,000 - 5,000 ops/; cache age is rarely above 3--most often 1 or 0. Consistency points are usually at the maximum of once every 10 seconds (as measured by disk writes with `sysstat`).
I was poking around on the SPEC pages, and in the footnotes to the NetApp entries are these parameters:
options nosnap 1 # to disable periodic snapshot creation, for reproducibility
options nosnapdir 1 # to avoid inserting .snapshot entry when reading directories
options raid.scrub.enable off # to disable periodic RAID scrubs, for reproducibility
options minra 1 # to minimize file read-ahead
options udp_lg_dgram.xmit_cksum.offload 1 # to offload checksum computations onto the Gigabit NIC
Openboot settings on the F740: setenv java? false # to disable the Java Virtual Machine
http://www.spec.org/osg/sfs97/results/res98q3/sfs97-980805-00008.html http://www.spec.org/osg/sfs97/results/res99q2/sfs97-19990416-00045.html
I have a few questions:
1a. Regarding snapshots, many of my development sandboxes are "disposable" so I don't care about snapshots. The first two options might help me.
Yes, if you don't care about snapshots, you should turn it off with those options.
In addition, if I set `snap reserve volX 0`, will that actually turn off snapshots, and reduce overhead on the filer? The options only seem to turn off *automatic* snapshots and the *display* of the ".snapshot" directory.
No, that just sets the snapshot only reserve to 0. The snapshot reserve is a low-water mark, not a high-water mark; snapshots can and will exceed it, otherwise that space is reserved and not used by the filesystem. Note that snapshots will still be created for things like backups (which I think still work with nosnapdir on, but that's something to check).
1b. Would disabling these two options *prevent* me from `cd`ing into the .snapshot directory? Even though it is visible by default at the root of the mount point, you can `cd` into the directory ".snapshot" at any point:
I think it would, but I confess I'm not sure from the description given in the technical papers.
1c. In addition, I'm thinking of breaking up my 14-disk filer from a RAID 13d+1p configuration into, say, two RAID groups and two volumes as 3d+1p and 9d+1p. (Multiple RAID groups are allowed per volume; I assume multiple volumes per RAID group is still disallowed?) The former would have the stuff I need snapshotted and the latter the stuff I don't. I have a churn rate of about 30-40%. Thoughts?
You'd have to set the options differently for each volume. I'm not sure that this would help you very much; if you have stuff that you want snapshotted, it's probably the stuff that gets accessed a lot.
The "no snapshot" stuff's impact on performanc is EXTREMELY small. I doubt you will even notice it, especially if you are not going "whole hog" with turning it off.
- What does "options minra 1" buy me? What is the number when
minra=off, the default? (If it's just a boolean, nevermind!)
It's just a boolean. This means the filer will do minimal readahead.
Given I have a development environment with lots of small files, it seems as if turning this option on will benefit me a lot. However, some more information on this would help.
It might. It might not. Best thing to do is turn it on and see. You can always turn it back off later if it hurts.
- Does the udp_lg_dgram.xmit_cksum.offload option also apply to
quad-ethernet cards? Will it help if the ports are trunked with a virtual interface?
No, I don't think so. In fact, to improve your performance you might consider a gigabit card rather than using the trunked ethernet.
- Why is my filer running Java?
The Java engine is used for many auxilliary functions.
If I disable that, would we see any administration impact? (These filers are administered solely via the
CLI.)
I do see the messages file logs
Yes, it would. What all would die when Java is disabled is unclear; I haven't seen a complete list, and it could change with each release.
To prevent a knee-jerk reaction, for the purposes of this discussion, upgrading my filer to an F760 is not an option. ;) I want to leverage what I have now, as best I can.
Your best way to leverage what you have now is to ask for a discount on the F760 for the upgrade. The second best way is to just buy a new filer and move some of the data over to it, using both.
Bruce