This is an F540 running 5.2.1P2D14 The filer is basically doing nothing - but every 5 seconds or so the CPU goes up to 80 or 90% and stays there for 6 or 7 seconds.
Besides the CPU anomaly, the cache age is awfully low for a filer which is "basically doing nothing." (It's a bit hard to see if the lines are wrapped in your message; I unwrapped them below.)
The fact that there is a flurry of disk writes in the midst of this CPU activity is interesting. A CP (signified by the writes) will use some CPU, but that should stop by the time of the last write. Most likely having the CP in the middle of these events is a coincidence.
Perhaps more interesting is that in this small sample the CPU usage coincides with some CIFS activity. The low network usage suggests this is some sort of meta operation such as lock fiddling. I would look at a larger sample and, if this correlation remains in a larger sample, I'd look at what was happening with CIFS. Possibly there is some operation which causes the filer to look at a large amount of cached data (accounting for the low cache age as well as CPU usage) without actually transferring much over the wire. It's been a long time since I looked at CIFS, though.
sysstat 1:
CPU NFS CIFS HTTP Net kB/s Disk kB/s Tape kB/s Cache in out read write read write age 1% 3 0 0 1 0 0 0 0 0 2 1% 2 0 0 0 0 0 0 0 0 2 2% 30 0 0 59 109 0 0 0 0 2 81% 26 12 0 16 148 0 0 0 0 2 90% 5 12 0 5 2 0 0 0 0 2 90% 3 12 0 4 2 0 0 0 0 2 92% 3 9 0 3 2 800 136 0 0 2 89% 2 11 0 4 2 16 732 0 0 2 90% 3 12 0 13 2 0 0 0 0 2 90% 2 13 0 4 2 0 0 0 0 2 13% 2 1 0 0 0 0 0 0 0 2 1% 10 1 0 2 2 0 0 0 0 2 1% 2 0 0 0 0 0 0 0 0 2 1% 4 0 0 9 1 0 0 0 0 2 1% 2 0 0 0 0 0 0 0 0 2 80% 3 12 0 4 2 0 0 0 0 2 90% 2 12 0 4 2 0 0 0 0 2 92% 1 12 0 4 2 312 0 0 0 2 93% 2 12 0 4 2 16 668 0 0 2 90% 4 12 0 4 2 0 0 0 0 2 CPU NFS CIFS HTTP Net kB/s Disk kB/s Tape kB/s Cache in out read write read write age 89% 4 11 0 4 2 0 0 0 0 2 92% 5 10 0 4 2 0 0 0 0 2 1% 10 0 0 2 2 0 0 0 0 2 1% 2 0 0 0 0 0 0 0 0 2 1% 9 0 0 1 1 0 0 0 0 2
-- Karl