We have used Netapp filers for 8 years and are very pleased. We currently have a F820c pair for NFS and CIFS access to home directories and web content. I am one of the primary filer admins.
We are about to purchase some disk storage to replace a lot of locally attached and SAN RAID arrays (Sun T3s and T4s) for some Solaris servers. Naturally I think we should buy Netapp.
One project is our administrative Oracle servers. I realize that Netapp and Oracle both recommend using NFS with Netapp, but we currently have a SAN infrastructure and lots of other inertia to overcome. If we buy Netapp for this, the filers will at least start out serving LUNs on our SAN.
I am convinced that Netapp is they way to go for NFS, but I have not seen much about Netapp in a SAN environment. We were looking at EMC Symmetrix, but probably cannot afford it, so it will probably be a choice between EMC Clariion and Netapp. Our "production instance" (I'm not familiar with the particulars) is 500G and growing. We also have a data warehouse instance and other misc. databases. The warehouse is copied fresh from production each night and then a 6 hour job crunches on it before it is made available for querying. Obviously a cloned volume would be nice here.
Is anyone running Oracle on Solaris over a SAN to a Netapp? How is the performance? How many filers and what models? How much storage? How many disks?
We are familiar with all the nifty snapshot features, volume cloning, etc. We just don't want to find out the hard way that the Netapp SAN solution is a poor performer with Oracle.
We also have a University email system on Solaris servers running Communigate Pro. We have over 20,000 users and about 1TB of email on disk. We are using mbox format and most email clients use imap. Our email admins looked into maildir format but found that in our environment it took ten times longer for imap to scan inboxes. This is not a surprise since inboxes usually have a lot of short messages, so it is often faster to just read the whole inbox than open a file for each message and read just the headers. And imap regularly needs to scan all the message headers in an inbox.
There are many advantages to the maildir format, but we are afraid that it would crush our current system. However, we are thinking that a Netapp filer would do a better job with maildir than a local Veritas filesystem on Solaris.
Is anyone out there running a large email system over NFS to a Netapp filer using maildir format? If you are using Communigate Pro, and/or do a lot of imap, that would be even better. How many email accounts and how much mail do you process? How many filers and what models? How much storage? Do you have any performance issues?
Currently our email servers do a lot more reads than writes (we read over 5 bytes for each byte we write). We don't know what the actual disk activity is inside the T3s and T4s, we only know how much we read and write from each LUN. We don't know the actual RAID 5 write overhead, and we don't know how many reads are satisfied from cache internal to the T3s and T4s.
Our email admins say that the T3s occasionally get swamped, but that the T4s seem to be doing fine, so they could probably replace the T3s with T4s and be OK for a couple years. But they want to expand storage and triple user quotas and perhaps switch to maildir, and they are afraid that T4s can't handle all that. Plus snapshots would make restoring lost email messages so much easier.
Thanks for any information you can give me. I'm going to look at Netapp's white papers, too.
Steve Losen scl@virginia.edu phone: 434-924-0640
University of Virginia ITC Unix Support
Steve, about 2 yrs ago we had one University mail system for both staff and students. It consisted of about 40K accounts using UW-IMAP with mbox style folders and one monolithic spool. User authentication was done via NIS+. One NetApp F840 contained qtree for the spool area (some 100+ GB) and another NetApp F840 contained qtrees for each of the staff and student home directories. Mail was delivered to home directories via sendmail/procmail.
This environment got to the point of being barely usable. The biggest problem for us that we experienced was a huge load on the filers due to file locking.
Two years ago in May, we reorganized the system to Courier/IMAP with maildir style folders for the students, about 36K accounts. A separate volume/qtree for the student mail folders and a separate volume/qtree for the student home directory.
Unfortunately at the last moment we were stopped to do the same for staff. The present staff mail system consists of 6K accounts and still uses the original set up, ie. UW-IMAP and one monolithic spool area all on a dedicated NetApp F840.
Needles to say the student system is flying; the filer usage is about 20 % CPU. The staff system has had the other filer pegged at 100 % for about two yrs now and mail response is terrible.
The entire mail system consists of: 5 delivery machines 5 postoffice machines (staff) 3 postoffice machines (students) 2 mailrelay machines all are running Debian Linux; a mixture of Dell 1600/1650 and Sun V60x
Don't be afraid to switch to maildir, it is fast ! It completely eliminated file locking for us. One thing, though, make sure that you come up with some sort of a good hash for the user accounts when creating the space for them. I have a two level structure with the top level going from 0-f and each of the sub-entries going from 00-ff.
Now there is a rumor that we will be converting the staff users to Courier this summer as they can no longer read mail ;-)
George
------------------------------------------------------------------------------- George Kahler e-mail: george@yorku.ca Sr. Systems Administrator humans: (416) 736-2100 x.22699 Computing and Network Services machines: (416) 736-5830 Ontario, Canada, M3J-1P3
On Wed, 23 Feb 2005 08:25:41 -0500, Steve Losen scl@sasha.acc.virginia.edu wrote:
We have used Netapp filers for 8 years and are very pleased. We currently have a F820c pair for NFS and CIFS access to home directories and web content. I am one of the primary filer admins.
We are about to purchase some disk storage to replace a lot of locally attached and SAN RAID arrays (Sun T3s and T4s) for some Solaris servers. Naturally I think we should buy Netapp.
One project is our administrative Oracle servers. I realize that Netapp and Oracle both recommend using NFS with Netapp, but we currently have a SAN infrastructure and lots of other inertia to overcome. If we buy Netapp for this, the filers will at least start out serving LUNs on our SAN.
I am convinced that Netapp is they way to go for NFS, but I have not seen much about Netapp in a SAN environment. We were looking at EMC Symmetrix, but probably cannot afford it, so it will probably be a choice between EMC Clariion and Netapp. Our "production instance" (I'm not familiar with the particulars) is 500G and growing. We also have a data warehouse instance and other misc. databases. The warehouse is copied fresh from production each night and then a 6 hour job crunches on it before it is made available for querying. Obviously a cloned volume would be nice here.
Is anyone running Oracle on Solaris over a SAN to a Netapp? How is the performance? How many filers and what models? How much storage? How many disks?
We are familiar with all the nifty snapshot features, volume cloning, etc. We just don't want to find out the hard way that the Netapp SAN solution is a poor performer with Oracle.
We also have a University email system on Solaris servers running Communigate Pro. We have over 20,000 users and about 1TB of email on disk. We are using mbox format and most email clients use imap. Our email admins looked into maildir format but found that in our environment it took ten times longer for imap to scan inboxes. This is not a surprise since inboxes usually have a lot of short messages, so it is often faster to just read the whole inbox than open a file for each message and read just the headers. And imap regularly needs to scan all the message headers in an inbox.
There are many advantages to the maildir format, but we are afraid that it would crush our current system. However, we are thinking that a Netapp filer would do a better job with maildir than a local Veritas filesystem on Solaris.
Is anyone out there running a large email system over NFS to a Netapp filer using maildir format? If you are using Communigate Pro, and/or do a lot of imap, that would be even better. How many email accounts and how much mail do you process? How many filers and what models? How much storage? Do you have any performance issues?
Currently our email servers do a lot more reads than writes (we read over 5 bytes for each byte we write). We don't know what the actual disk activity is inside the T3s and T4s, we only know how much we read and write from each LUN. We don't know the actual RAID 5 write overhead, and we don't know how many reads are satisfied from cache internal to the T3s and T4s.
Our email admins say that the T3s occasionally get swamped, but that the T4s seem to be doing fine, so they could probably replace the T3s with T4s and be OK for a couple years. But they want to expand storage and triple user quotas and perhaps switch to maildir, and they are afraid that T4s can't handle all that. Plus snapshots would make restoring lost email messages so much easier.
Thanks for any information you can give me. I'm going to look at Netapp's white papers, too.
Steve Losen scl@virginia.edu phone: 434-924-0640
University of Virginia ITC Unix Support
-
George Kahler -
Steve Losen