On Fri, 6 May 2005, Darren Dunham wrote:
We are mounting storage for a (brain dead) application which stores files generated daily on a Filer Volume. The dir has 125,000 files and growing.
The mount point is on a Solaris 2.8 box. What is the limit on number of files in a folder in this context? Anybody know?
I don't know any limit here (either for UFS or NFS). Obviously you start reducing performance on access because there's no file index in the directory, but the lack of an index also means that there's no static table to overflow here.
On UFS2, the maximum number of files is 1,000,000. UFS2 includes FFS which provides indexing of the directory. Accessing files in a directory which that many files is was not too bad in our experience using a single drive under FreeBSD, it did it in less than 1 second. What was very slow was to delete files, about 16 seconds per file.
I worked out that if your average file size is greater than 8KB you will run out of disk space before you run out of inodes going by the default inode count set by newfs.
default fragment size is 2048bytes, by default it creates one inode for every 4 fragsize. hence the 8KB i mention earlier.
One thing you can do is set the number of inodes with newfs -i, the default being 8196. I guess you could go down to 2048, I am not sure though what would happen if you go below that figure.
So first work out your average file size... and then decide how to newfs the drive. If you are not using a netapp and you are using a raid controller that allows you to set the stripe size on a volume you will get better performance if you also match the stripe size to the average file size.
A word of caution though. If you think that the average file size is likely to increase in the near future or you are close to the boundary between tripe sizes, use next stripe size. Eg, your average file size is 30KB a strip of 32KB maybe best at present. If the average file size increases over 32KB chose 64KB... of course makes sure that the average is based on the majority of the files, make your average is not pushed over the boundary because you have 1 file which is 4GB or something.
Anyway this is all pretty simplistic compared to how the netapp manages WAFL and stripe sizes. I can see that an admin can increase the inode on a file system. I guess that then re-structures the file system on the fly. have a read. http://www.netapp.com/tech_library/3002.html
Regards, Maren.
You might have to increase maxdir size on the filer at some point. I don't know how many files the standard limit will allow.
-- Darren Dunham ddunham@taos.com Senior Technical Consultant TAOS http://www.taos.com/ Got some Dr Pepper? San Francisco, CA bay area < This line left intentionally blank to confuse you. >
-------------------------------------------------------------- HKdotCOM Ltd Tel: 852 2865-4865 ext 888 Fax: 852 2865-4100 leizaola@hk.com AIM: MarenHKdotCOM ICQ: 39905706 MSN:MarenHKdotCOM -------------------------------------------------------------- Get your @hk.com email address on http://www.hk.com