Hi all, are there any limits to the maximum number of files (or subdirectories) a Netapp NFS mounted directory can handle?
In our case (if it matters) we are looking at Linux 2.4.18 using NFS v3/tcp to a Netapp running 6.1.2R3.
We are thinking of a directory with 500,000+ files in it.
Cheers, CM
chris@psychofx.com (Chris Miles) writes
Hi all, are there any limits to the maximum number of files (or subdirectories) a Netapp NFS mounted directory can handle?
In our case (if it matters) we are looking at Linux 2.4.18 using NFS v3/tcp to a Netapp running 6.1.2R3.
We are thinking of a directory with 500,000+ files in it.
Look at "options wafl.maxdirsize" (or, in more recent ONTAP versions, "vol options maxdirsize"). The default is 10 MB, and a directory that size will hold 327680 entries if the names are small enough: less than that otherwise.
You can increase the value, but directories this size are not really a very good idea, even with the ONTAP fast lookup enhancements.
Whether the Linux client matters depends on what it thinks of caching NFS readdir results of that size. Not with great enthusiasm, I would guess. :-)
Does your application really need a flat directory structure this wide?
Chris Thompson Email: cet1@cam.ac.uk
On Wed, Aug 28, 2002 at 06:03:38PM +0100, Chris Thompson wrote:
Look at "options wafl.maxdirsize" (or, in more recent ONTAP versions, "vol options maxdirsize"). The default is 10 MB, and a directory that size will hold 327680 entries if the names are small enough: less than that otherwise.
Great, thanks.
You can increase the value, but directories this size are not really a very good idea, even with the ONTAP fast lookup enhancements.
Whether the Linux client matters depends on what it thinks of caching NFS readdir results of that size. Not with great enthusiasm, I would guess. :-)
Does your application really need a flat directory structure this wide?
Well that is being debated, but at least I know it is technically possible. If it gets to that we'll do some performance testing.
Cheers, CM
I remember we had a problem with Linux (probably RedHat 7.1) where we could only access 32767 files before things started failing. I believe we found it was a limitation on the Linux side, not a NetApp issues.
-dale ----- Original Message ----- From: "Chris Miles" chris@psychofx.com To: toasters@mathworks.com Sent: Wednesday, August 28, 2002 1:00 PM Subject: limit to number of files in a directory
Hi all, are there any limits to the maximum number of files (or subdirectories) a Netapp NFS mounted directory can handle?
In our case (if it matters) we are looking at Linux 2.4.18 using NFS v3/tcp to a Netapp running 6.1.2R3.
We are thinking of a directory with 500,000+ files in it.
Cheers, CM
-- Chris Miles chris@psychofx.com http://www.psychofx.com/chris/
We have noticed similar problems with large directories on a NetApp filer. The resolution involved emptying the directory, removing it and recreating it. That brings performance back to normal.
No changes were on the client side so we have concluded that the issue was with the Filer.
--Walter
Dale Gass wrote:
I remember we had a problem with Linux (probably RedHat 7.1) where we could only access 32767 files before things started failing. I believe we found it was a limitation on the Linux side, not a NetApp issues.
-dale ----- Original Message ----- From: "Chris Miles" chris@psychofx.com To: toasters@mathworks.com Sent: Wednesday, August 28, 2002 1:00 PM Subject: limit to number of files in a directory
Hi all, are there any limits to the maximum number of files (or subdirectories) a Netapp NFS mounted directory can handle?
In our case (if it matters) we are looking at Linux 2.4.18 using NFS v3/tcp to a Netapp running 6.1.2R3.
We are thinking of a directory with 500,000+ files in it.
Cheers, CM
-- Chris Miles chris@psychofx.com http://www.psychofx.com/chris/
-
Chris Miles -
Chris Thompson -
Dale Gass -
Walter Ellinthorpe