Hi Adi,
We're running a few 9i RAC clusters. Not a lot in production yet, but we've been stressing and tuning it for over a year. Typically 4 nodes per cluster, using a combination of OCFS/iSCSI and NFS. Datafiles go onto OCFS/iSCSI and archive logs go onto NFS (because OCFS sucks at small files).
We're using Redhat 2.4.9-e.38enterprise. We've been working directly with Netapp and Oracle development, pulling them tooth and nail along the cert process, and I believe it just got completed, so we're probably the first in production. Contact Oracle for more info.
We even have a bi-coastal set of clusters, with multiple masters running streams replication; as far as I know we're the first at that too. It's difficult to implement, I wouldn't recommend it.
Note that we are using the Adaptec 7211C TOEs, not the Cisco drivers.
Oh, and by the way, it works great, and many of our NFS DBA's are drooling over iSCSI at this point. It's much faster than NFS, and much cheaper than FCP, and when 10gig comes out, it will be much faster *and* cheaper than either.
There are some Adaptec tuning parameters I can share if people are using the 7211C's. It helps to raise the inflight cap. Also, there are some hacks you can do to force linux to mount multiple luns. By default it only scans the first lun per target (because of a problem with CD stackers trying to scan every cd at boot). The Adaptec instructions tell you how to rebuild the kernel, which, of course, defeats your Oracle support. There is a better way...
The downside is that you don't get the great features that have been baked into Netapp's NAS software for so long. Remember that iSCSI is SAN, and Netapp rocks at NAS, but are new at SAN. Snapshots take up a lot of space, a full lun size per snap or +1 lun + 20%,depending on who you talk to. Also, the nice .snapshot directory isn't there. You can implement that with cron jobs quiescing the db, deleting/creating/renaming/remounting snapped luns, but it's messy.
Bottom line is that if you're considering a move from NFS, the performance will blow you away, but the maintenance might be a pain in the arse.
-Mike Christian
-----Original Message----- From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Aditya Sent: Friday, June 11, 2004 10:44 AM To: toasters@mathworks.com Subject: oracle 9i cluster using iScsi?
Anyone have experience(s) running an Oracle 9i cluster with data stored via iScsi? on a netapp? with Solaris or RedHat?
We already run iScsi for other things and have no experience (or interest) in DAFS just for Oracle. We're trying to replace Sun 3310 RAID arrays with network-attached storage...
Thanks, Adi