Is this really, possibly, wonderfully true??!! Which version of ONTAP fixed this? The reason I ask is that on all our filers (mostly F760 and some F840's), the total number of folders at any one directory level was a function of the inode limit of a 2 bit, unsigned pointer - thus limited it to just under 65k directories at any one level. Since we have 9.5 million user directories spread across those filers, we had to write scripts to check when that threshold was being reached to create a new one.
We would just LOVE to hear that this limit has been removed!
-----Original Message----- From: Bruce Sterling Woodcock [mailto:sirbruce@ix.netcom.com] Sent: Tuesday, December 19, 2000 10:41 AM To: neil lehrer; toasters Subject: Re: 2 questions for f760
----- Original Message ----- From: "neil lehrer" nlehrer@ibb.gov To: "toasters" toasters@mathworks.com Sent: Tuesday, December 19, 2000 9:25 AM Subject: 2 questions for f760
hi,
- we will be moving our home and share dirs from old sun boxes to
f760. the home dirs on the suns were set up like:
\home \home\a \home\a\abe \home\a\allen \home\b
etc. in other words, user homedirs under the first letter of the id. i was told this was done for performance reasons, inode searching, on the sun boxes.
do i need to use this structure on the 760? is there a similar issue on filers? or can it just be \home\userid?
Unless you're talking 100,000 entries or so, it can be \home\userid. NTAP implemented a large directory performance enhancement a long time ago that could handle 10s of thousands of entries, and the 760 is many, many times faster than that. If you want to read the old paper, though:
http://www.netapp.com/tech_library/3006.html
Given the filesystem enhancements since then they may even have gone to b-trees by now.
Bruce
----- Original Message ----- From: "Sam Schorr" sschorr@homestead-inc.com To: "'Bruce Sterling Woodcock'" sirbruce@ix.netcom.com; "neil lehrer" nlehrer@ibb.gov; "toasters" toasters@mathworks.com Sent: Tuesday, December 19, 2000 11:45 AM Subject: RE: 2 questions for f760
Is this really, possibly, wonderfully true??!! Which version of ONTAP
fixed
this?
A *long* time ago.
The reason I ask is that on all our filers (mostly F760 and some F840's), the total number of folders at any one directory level was a function of the inode limit of a 2 bit, unsigned pointer - thus limited it to just under 65k directories at any one level. Since we have 9.5 million user directories spread across those filers, we had to write scripts to check when that threshold was being reached to create a new one.
This is something different. I was not aware of the 65,000 limit per directory and the paper in question only gives data up to 60,000 so I was just extrapolating.
Bruce
On Tue, 19 Dec 2000, Sam Schorr wrote:
Is this really, possibly, wonderfully true??!! Which version of ONTAP fixed this? The reason I ask is that on all our filers (mostly F760 and some F840's), the total number of folders at any one directory level was a function of the inode limit of a 2 bit, unsigned pointer - thus limited it to just under 65k directories at any one level. Since we have 9.5 million user directories spread across those filers, we had to write scripts to check when that threshold was being reached to create a new one.
9.5 million!!! Jeepers.
A quick sh script verifies 2^16 - 1 directory entries are possible, no more:
mkdir: Failed to make directory "65533"; Too many links
(0-65532 directories, plus "." and ".." equals 65,535)
It's exactly linear; every 128 entries, the size of the directory grows by 4,096 bytes. Cool. Linearity is good, and predictable.
Until next time...
The Mathworks, Inc. 508-647-7000 x7792 3 Apple Hill Drive, Natick, MA 01760-2098 508-647-7001 FAX tmerrill@mathworks.com http://www.mathworks.com ---
----- Original Message ----- From: "Todd C. Merrill" tmerrill@mathworks.com To: "Sam Schorr" sschorr@homestead-inc.com Cc: "toasters" toasters@mathworks.com Sent: Tuesday, December 19, 2000 3:04 PM Subject: RE: 2 questions for f760
On Tue, 19 Dec 2000, Sam Schorr wrote:
Is this really, possibly, wonderfully true??!! Which version of ONTAP
fixed
this? The reason I ask is that on all our filers (mostly F760 and some F840's), the total number of folders at any one directory level was a function of the inode limit of a 2 bit, unsigned pointer - thus limited
it
to just under 65k directories at any one level. Since we have 9.5
million
user directories spread across those filers, we had to write scripts to check when that threshold was being reached to create a new one.
9.5 million!!! Jeepers.
A quick sh script verifies 2^16 - 1 directory entries are possible, no more:
mkdir: Failed to make directory "65533"; Too many links
(0-65532 directories, plus "." and ".." equals 65,535)
It's exactly linear; every 128 entries, the size of the directory grows by 4,096 bytes. Cool. Linearity is good, and predictable.
Of course, that's different from wafl.maxdirsize, which limits the parent directory to a size of 10MB, which should be able to hold 327,680 such entries....
So, obviously, the directory limitation is different, and I am actually kinda surprised to see it is still there... time for a new filesystem format? :)
Bruce
Is this really, possibly, wonderfully true??!! Which version of ONTAP fixed this? The reason I ask is that on all our filers the total number of folders at any one directory level was a function of the inode limit of a 2 bit, unsigned pointer
... and still is, AFAIK. We can cope with lots of files, just not lots of subdirectories. That said, I haven't met a million-file or million-directory application yet which wasn't served better by bushing things up a bit and keeping intermediate subdirectories smaller.
Bruce added:
So, obviously, the directory limitation is different, and I am actually kinda surprised to see it is still there... time for a new filesystem format? :)
That's what it'd take. There's just not that much spare room in the inode to hold a bigger link count.
Regards, Garth.
-- Garth T Kidd garth@netapp.com Mobile: +61-411-596-593 Consulting Systems Engineer, Asia Pacific Mobile Fax: +61-3-9228-9732 Network Appliance http://www.netapp.com/
-
Bruce Sterling Woodcock -
Garth Kidd -
Sam Schorr -
Todd C. Merrill