On Tue, 17 Jul 2001, Todd C. Merrill wrote:
We turn off NFS to see if that is the culprit. Still CPU is pegged. We terminate CIFS to see if that is. CPU drops down, but not to zero. We have only NFS, CIFS, and cluster licensed (i.e., no HTTP).
Here's what we saw:
home> sysstat 2 CPU NFS CIFS HTTP Net kB/s Disk kB/s Tape kB/s Cache in out read write read write age 10% 0 0 0 10 6 12074 0 0 0 24 47% 0 0 0 8 5 6369 7 0 0 24 47% 0 0 0 13 7 16479 20258 0 0 24 33% 0 0 0 4 3 11858 13180 0 0 24 34% 0 0 0 8 4 12698 14703 0 0 24
As a follow-up to this, we never found out why the filer stayed mental after all services were turned off, but we found what made it *start* to go mental. A bad entry in our WINS database caused the filer to try to authenticate against either non-existent or topologically distant domain controllers. The CIFSAuthen process on the filer showed it taking up most of the CPU resources (50-60%), apparently in a wait state, waiting for a timeout or waiting for the slow response from a distant DC. Removing the bad/distant entries has apparently left the filer with a closer, faster set of DC's against which to choose to authenticate.
If you'll recall, this problem was with ONTAP 5.3.7. We have since found out that in ONTAP 6.x, there is the command:
cifs prefdc
to give the filer a hard-coded list of (fast and local) DC's against which to authenticate, similar to the `options nis.servers` for NIS servers. We are now accelerating our upgrade plan to take advantage of this feature. ;)
Thanks to all who replied publically and privately with suggestions. Thanks to the NetApp admin class 202 for all kinds of cool rc_toggle_basic goodies. And, thanks to the NetApp tech support folks and our local SE for the interpretation of all of this data. It was, in the best sense of the word, a team effort in nailing this problem down from the deceptive symptoms.
Until next time...
The Mathworks, Inc. 508-647-7000 x7792 3 Apple Hill Drive, Natick, MA 01760-2098 508-647-7001 FAX tmerrill@mathworks.com http://www.mathworks.com ---