What is the best way to monitor large directory sizes? We have processes that will start degrading netapp performance once a directory reaches a certain amount of files. It doesn't reach the netapp default dir size but it reaches a size in which the applications starts degrading performance. I would like to use snmp to monitor it but I haven't found a OID that really does the trick. Can anyone share their experiences with monitoring large dirs?
Thanks
-Mark
markallen> What is the best way to monitor large directory sizes? We markallen> have processes that will start degrading netapp performance markallen> once a directory reaches a certain amount of files. It markallen> doesn't reach the netapp default dir size but it reaches a markallen> size in which the applications starts degrading markallen> performance. I would like to use snmp to monitor it but I markallen> haven't found a OID that really does the trick. Can anyone markallen> share their experiences with monitoring large dirs?
I think you're out of luck, since NetApp doesn't offer SNMP down to that level of granularity. The best I would expect to see is on a per-qtree and per-volume basis if you have quotas enabled.
In your case, the best bet is going to be some tool which periodically looks at the directory and grabs the number of files and sends you a warning if they're too high. This could be done on another host which has an SNMP daemon which allows callouts to a script and can return the result (number of files in the directory) at a custom OID you specify.
The real solution is to fix you application to spread files across more sub-directories using some sort of hashing algorithm. The simplest one would be to just create directories of the first letter (and or number if needed) of the filename and put the files down a level.
Somehow I bet your app won't let you do that. Look at the net-snmp package for how to write snmp client monitoring scripts, I'm sure there's something in there which you can hack to make work in your environment.
John
Found something interesting looking throught the MIB today:
I think this might help...
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- This section contains traps related to alerting when 64MB directory -- limit is nearly reached or has been reached. -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
waflDirFull TRAP-TYPE ENTERPRISE netapp DESCRIPTION "The directory has been filled to its limit"
::= 187
-----Original Message----- From: owner-toasters@mathworks.com on behalf of John Stoffel Sent: Mon 6/12/2006 10:27 PM To: markallen@micron.com Cc: toasters@mathworks.com Subject: Re: Monitor large directory sizes
markallen> What is the best way to monitor large directory sizes? We markallen> have processes that will start degrading netapp performance markallen> once a directory reaches a certain amount of files. It markallen> doesn't reach the netapp default dir size but it reaches a markallen> size in which the applications starts degrading markallen> performance. I would like to use snmp to monitor it but I markallen> haven't found a OID that really does the trick. Can anyone markallen> share their experiences with monitoring large dirs?
I think you're out of luck, since NetApp doesn't offer SNMP down to that level of granularity. The best I would expect to see is on a per-qtree and per-volume basis if you have quotas enabled.
In your case, the best bet is going to be some tool which periodically looks at the directory and grabs the number of files and sends you a warning if they're too high. This could be done on another host which has an SNMP daemon which allows callouts to a script and can return the result (number of files in the directory) at a custom OID you specify.
The real solution is to fix you application to spread files across more sub-directories using some sort of hashing algorithm. The simplest one would be to just create directories of the first letter (and or number if needed) of the filename and put the files down a level.
Somehow I bet your app won't let you do that. Look at the net-snmp package for how to write snmp client monitoring scripts, I'm sure there's something in there which you can hack to make work in your environment.
John
-
Glenn Walker -
John Stoffel -
markallen＠micron.com