We're considering purchasing filer(s) as part of a mail system upgrade. Setup would be a cluster of redundant and very similarly configured machine around one or more filers containing all the real data (/var/spool/mail for customers, some web serving, and other stuff). Most likely all the clustered machines will do delivery to the shared area and POP/IMAP reading.
Our desired platform is PCs running FreeBSD (or perhaps BSDI). One major problem is that none of the PC Unix environments (FreeBSD, BSDI, Linux) have working NFS file locking. We could implement some hacks using file based locking, but I'm worried about the hassle and performance issues here.
In fact, even with working NFS locking, I'd be worried about sharing mail spools in this fashion.
I've also had advice from people who recommend avoiding NFS entirely and going for transparent proxies and multiple machines for scalability. But that leaves a reliability/redundancy issue (hardware RAIDs, s/w mirroring, ...).
What I'm wondering and perhaps people on the list can help is: - anyone doing something similar in an ISP environment (what platform, what filer, how many users?) - any advice/ideas on the locking issue - will filers perform well in this environment? some of the client applications are not well behaved and do things like copy the entire mail box; local disk would obviously be faster (I can get over 35MB/sec using multiple local disks on PCs), but I'm not sure it's that big an issue -- Alan Judge Phone: +353-1-6046901 Indigo Network Operations Centre Fax: +353-1-6046948
Alan Judge wrote:
We're considering purchasing filer(s) as part of a mail system upgrade. Setup would be a cluster of redundant and very similarly configured machine around one or more filers containing all the real data (/var/spool/mail for customers, some web serving, and other stuff). Most likely all the clustered machines will do delivery to the shared area and POP/IMAP reading.
Our desired platform is PCs running FreeBSD (or perhaps BSDI). One major problem is that none of the PC Unix environments (FreeBSD, BSDI, Linux) have working NFS file locking. We could implement some hacks using file based locking, but I'm worried about the hassle and performance issues here.
In fact, even with working NFS locking, I'd be worried about sharing mail spools in this fashion.
Even sun have had problems getting the locking code right, I've heard...
I've also had advice from people who recommend avoiding NFS entirely and going for transparent proxies and multiple machines for scalability. But that leaves a reliability/redundancy issue (hardware RAIDs, s/w mirroring, ...).
What I'm wondering and perhaps people on the list can help is:
[snip]
- any advice/ideas on the locking issue
It all depends on whether you mean "shared mail queue" or "shared maildrops"... Sharing the mail queue is a big nono, really. None of the existing MTA's support doing this, and it will cause you slow mail delivery and general havoc.
Shared maildrops are a different issue. If you use something like qmail[1], OTOH, there are simply no locking issues whatsoever (and easy POP3 support, but no IMAP). We do something similiar (although propietary) for about 100,000 users, and it works well. Don't forget to set up a directory hashing scheme, though as 100,000 files in a directory will also cause you performance problems. :-)
-Dom
Our clients are 80+ HP workstations, few SGI and a couple of Convex. Their /usr/mail (/var/mail) are NFS mounted (automount) to FAServer450 for the last ~2years. So far its working. This is the permissions on the mail directory on the FAServer: # ls -ld mail drwxrwxrwt 3 root mail 8192 Jul 2 10:01 mail
-- Begin original message --
From: Alan Judge Alan.Judge@indigo.ie Date: Wed, 02 Jul 1997 17:04:06 +0100 Subject: Filers for a large mail environment To: toasters@mathworks.com
We're considering purchasing filer(s) as part of a mail system upgrade. Setup would be a cluster of redundant and very similarly configured machine around one or more filers containing all the real data (/var/spool/mail for customers, some web serving, and other stuff). Most likely all the clustered machines will do delivery to the shared area and POP/IMAP reading.
Our desired platform is PCs running FreeBSD (or perhaps BSDI). One major problem is that none of the PC Unix environments (FreeBSD, BSDI, Linux) have working NFS file locking. We could implement some hacks using file based locking, but I'm worried about the hassle and performance issues here.
In fact, even with working NFS locking, I'd be worried about sharing mail spools in this fashion.
I've also had advice from people who recommend avoiding NFS entirely and going for transparent proxies and multiple machines for scalability. But that leaves a reliability/redundancy issue (hardware RAIDs, s/w mirroring, ...).
What I'm wondering and perhaps people on the list can help is:
- anyone doing something similar in an ISP environment (what platform, what filer, how many users?)
- any advice/ideas on the locking issue
- will filers perform well in this environment? some of the client applications are not well behaved and do things like copy the entire mail box; local disk would obviously be faster (I can get over 35MB/sec using multiple local disks on PCs), but I'm not sure it's that big an issue
-- Alan Judge Phone: +353-1-6046901 Indigo Network Operations Centre Fax: +353-1-6046948
-- End original message --
Philip Thomas Motorola - ACT, M/S M350 2200 W. Broadway M350 Mesa, AZ 85202 thomas@act.sps.mot.com rxjs80@email.sps.mot.com (602) 655-3678 (602) 655-2285 (fax)
On Wed, 2 Jul 1997, Alan Judge wrote:
We're considering purchasing filer(s) as part of a mail system upgrade. Setup would be a cluster of redundant and very similarly configured machine around one or more filers containing all the real data (/var/spool/mail for customers, some web serving, and other stuff). Most likely all the clustered machines will do delivery to the shared area and POP/IMAP reading.
My understanding is that even with nfs locking having multiple systems do delivery is not safe.
Could you live with a single (+ backup) delivery system? Having several MXs of different preferences should be ok as I would expect only one to actually be delivering mail at any time.
Other comments?
joel
-
Alan Judge -
Dom Mitchell -
Joel Gallun -
Philip Thomas