These considerations still matter, but the number of files in a directory have to be pretty large for performance considerations to come into play. Typically speaking, splitting up the entries so that there are less per directory is still a good idea - but in this case, I'd say that only 200 per directory is too small and a bit of a waste.
What you're looking for is a ceiling at which the max entries will become too much to bear - that depends on the application. I would think keeping it less than 100,000 wouuld suffice for WAFL, but at some point the client will take an inordinate amount of time to enumerate all of the entries. Perhaps something more along the lines of 10,000 entries would be a good place to start?
Glenn
-----Original Message----- From: owner-toasters@mathworks.com on behalf of Page, Jeremy Sent: Wed 7/19/2006 12:27 PM To: David Lee; toasters@mathworks.com Subject: RE: directory efficiency
I think it might, as far as I know filers still use inodes, and it seems that the nested levels would be what would cause the problems here...but since they are going to increase the number of blocks then can address at the ^2 of the number of inodes deep I'm surprised this would cause performance issues. Adam probably can give you a 100% answer.
-----Original Message----- From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of David Lee Sent: Wednesday, July 19, 2006 10:37 AM To: toasters@mathworks.com Subject: directory efficiency
(Is the following an FAQ? If so, apologies, point me in the right direction...)
In years gone by, on a typical UNIX filesystem, creating entries (e.g. new files) in a directory could become badly inefficient as that directory size increased. By the time that (for instance) a "/var/spool/mail" reached (say) 20,000 entries, the acts of creating and deleting lockfiles in that directory could seriously impede things. (As a consequence, various sites devised various workaround; for instance we split into 100 subdirectories based on "uid mod 100", so that each subdir was only around (say) 200 entries.
When we migrated this to NetApp a couple of years back, we simply kept this subdivision in place. But now another consideration is coming into play which would be eased if we could consider reverting to the single large (~20,000 entry) directory.
Do such efficiency considerations matter in NetApp WAFL?