On Fri, 11 Feb 2005, Ben Rockwood wrote:
UFS (Solaris9/Solaris10) will cut it, but probly isn't the best without tuning, and definately not on Linux.
we are using UFS2 in a single directory and it is ok for reading and writing files but deleting them takes for ever.
From what I can see UFS2 is not much better than UFS for this application.
If your using Linux, carefully consider JFS and XFS,
The dynamic allocation of inodes is interesting....
We don't need journaling the data in the database is not critical..
I just found this... which is interesting... http://en.wikipedia.org/wiki/Comparison_of_file_systems
both can do what you want but have up and downsides. Do NOT run ReiserFS in production, too many people have been bitten by it. EXT2/3 are just not even contenders, and who wants to pay for VxFS. For your requirements my first attempt (on Linux anyway) would be JFS.
And, ZFS, btw, would handle this all nicely, but won't be introduced to Solaris10 untill Update1, sometime in the summer.
what a scam... they announced it was part of solaris 10... and it is not included??
maren.
benr.
-----Original Message----- From: owner-toasters@mathworks.com on behalf of Maren S. Leizaola Sent: Fri 2/11/2005 2:35 AM To: toasters@mathworks.com Cc: Subject: high performance file systems, non Netapp.
Hi, We have an application where we need to store between 4 to 20 million small files on the on a large drive or a 3 drive raid system.
We are finding that file systems like UFS just don't cut it never mind Linux extFS... These file systems can handle what we are trying to do but they tend to slow down as they get full and in some cases we will have to be able to delete millions of files in one go.
In this type of configuration unfortunately we can't bundle at netapp as the technical overhead and cost would make the system unfeasable.
I have so far seen the ReiserFS can do what we are looking for but we need a fs that will work on Linux, FreeBSD, Solaris etc... We must have a path to scale up or deploy a heavy load system and migrate to a big sun box if required.
Anyone knows how well Solaris 10's ZFS would work for this kind of application?
What do you guys suggest?
Maren.
HKdotCOM Ltd Tel: 852 2865-4865 ext 888 Fax: 852 2865-4100 leizaola@hk.com AIM: MarenHKdotCOM ICQ: 39905706 MSN:MarenHKdotCOM
Get your @hk.com email address on http://www.hk.com
-------------------------------------------------------------- HKdotCOM Ltd Tel: 852 2865-4865 ext 888 Fax: 852 2865-4100 leizaola@hk.com AIM: MarenHKdotCOM ICQ: 39905706 MSN:MarenHKdotCOM -------------------------------------------------------------- Get your @hk.com email address on http://www.hk.com