Has anyone got experience with running Oracle on NetApp filers? I am looking for information for a project I am involved in.
1. How does performance really stack up compared with a DAS or SAN solution? Does the IP stack -vs- SCSI stack cause a significant performance lag?
2. Are there compelling reasons why this would be a preferred solution?
3. Are there compelling reasons why this solution should be avoided?
4. What are the gotchas of such a solution? Are there configuration issues that can trip you up?
I've read the whitepapers on the NetApp and Oracle sites. I am looking for someone though that has actually done this.
Thanks,
Robin winslett@ev1.net
- How does performance really stack up compared with a DAS or SAN
solution? Does the IP stack -vs- SCSI stack cause a significant performance lag?
Robin,
It all depends on which Unix platform you run Oracle. Solaris' NFS client in any release before Solaris 9 has a huge bug that slows down performance. Try exporting a filesystem on your Solaris host and mounting localhost:/filesystem. Then try some benchmarks...they stink. As I understand it, the HP-UX nfs client does not have these issues and performs well with NAS.
- Are there compelling reasons why this would be a preferred solution?
Netapp features: instant snapshot, snapmirror, snaprestore. These features are critical in an Oracle (and Clearcase) implementation.
Are there compelling reasons why this solution should be avoided?
What are the gotchas of such a solution? Are there configuration issues that can trip you up?
NFS over UDP is sometimes recommended by Netapp SE's to avoid TCP overhead. TCP is preferable in my opinion.
I have not personally implemented Oracle on NAS, but I've been in contact with folks in my company who have played with it. The Solaris gotcha is the big problem. If you're using ORacle on HP-UX, try it out.
/Brian/
Brian,
In reagards to the Solaris NFS issue, I've heard it is the other way around. UDP with Solaris and the filers is a loser but TCP works well. I have actually experienced that myself and my SE said it was a known issue with Solaris, Netapp, and <insert switch vendor here>. It's only a problem with the Solaris NetApp combo though, NFS over UDP that is Solaris to Solaris worked fine for me.
And, since this has been opened to people who haven't actually implemented Oracle on NAS, the only real issues I've heard of are when people use automount maps for Oracle mounts. Sometimes they go away and Oracle doesn't like that. But I've heard from numerous sources that are running Oracle on a filer that if you hard mount it's a great solution.
ymmv, obviously.
~JK
Brian Long wrote:
- How does performance really stack up compared with a DAS or SAN
solution? Does the IP stack -vs- SCSI stack cause a significant performance lag?
Robin,
It all depends on which Unix platform you run Oracle. Solaris' NFS client in any release before Solaris 9 has a huge bug that slows down performance. Try exporting a filesystem on your Solaris host and mounting localhost:/filesystem. Then try some benchmarks...they stink. As I understand it, the HP-UX nfs client does not have these issues and performs well with NAS.
- Are there compelling reasons why this would be a preferred solution?
Netapp features: instant snapshot, snapmirror, snaprestore. These features are critical in an Oracle (and Clearcase) implementation.
Are there compelling reasons why this solution should be avoided?
What are the gotchas of such a solution? Are there configuration issues that can trip you up?
NFS over UDP is sometimes recommended by Netapp SE's to avoid TCP overhead. TCP is preferable in my opinion.
I have not personally implemented Oracle on NAS, but I've been in contact with folks in my company who have played with it. The Solaris gotcha is the big problem. If you're using ORacle on HP-UX, try it out.
/Brian/
Brian Long | | | Americas IT Hosting Sys Admin | .|||. .|||. Phone: (919) XXX-XXXX | ..:|||||||:...:|||||||:.. Pager: (888) XXX-XXXX | C i s c o S y s t e m s
On Tue, Aug 20, 2002 at 07:10:23AM -0700, Jeff Kennedy wrote:
Brian,
In reagards to the Solaris NFS issue, I've heard it is the other way around. UDP with Solaris and the filers is a loser but TCP works well. I have actually experienced that myself and my SE said it was a known issue with Solaris, Netapp, and <insert switch vendor here>. It's only a problem with the Solaris NetApp combo though, NFS over UDP that is Solaris to Solaris worked fine for me.
As far as I'm concerned, this is an urban legend :-) I use UDP v3 on Solaris with NetApp, and I've always had a good performance, no worse than Linux or HPUX. When you say you experienced it yourself, what exactly did you see? Was it reproducible with iozone? What were the numbers?
Thanks
Igor
Reproduceable on every Sun server running gigabit for 4 days. We upgraded code on the Foundry gear we use on a Monday and, wham!, all Sun gigabit performance went to hell. It was odd that the 100mb connections still worked as before.
I spoke with Foundry and NetApp as well as Sun. My local SE said "turn on TCP" since it was a know issue ( he was suprised support didn't tell me that ). Once we did that and remounted the filesystems under TCP it went back to normal.
Now I know what you're going to say; "it was the Foundry code idiot!". And yes, that is partially correct. But under that same Foundry code, before we turned TCP on on the filers, we did Sun to Sun NFS and those worked fine under UDP and TCP. My SE said it was specifically a Sun/NetApp/<Foundry?> issue.
As to why it worked under UDP pre-code upgrade I couldn't say.
~JK
Igor Schein wrote:
On Tue, Aug 20, 2002 at 07:10:23AM -0700, Jeff Kennedy wrote:
Brian,
In reagards to the Solaris NFS issue, I've heard it is the other way around. UDP with Solaris and the filers is a loser but TCP works well. I have actually experienced that myself and my SE said it was a known issue with Solaris, Netapp, and <insert switch vendor here>. It's only a problem with the Solaris NetApp combo though, NFS over UDP that is Solaris to Solaris worked fine for me.
As far as I'm concerned, this is an urban legend :-) I use UDP v3 on Solaris with NetApp, and I've always had a good performance, no worse than Linux or HPUX. When you say you experienced it yourself, what exactly did you see? Was it reproducible with iozone? What were the numbers?
Thanks
Igor
On Tue, Aug 20, 2002 at 10:47:13AM -0700, Jeff Kennedy wrote:
Reproduceable on every Sun server running gigabit for 4 days. We upgraded code on the Foundry gear we use on a Monday and, wham!, all Sun gigabit performance went to hell. It was odd that the 100mb connections still worked as before.
There are issues with flow control, back-to-back network packets and buffer space on switches. This is not related to Solaris boxes, Foundry switches or NetApp filers in particular, it's just a place where people notice the issues. I have experienced the same with Cisco switches.
Robin Winslett wrote:
Has anyone got experience with running Oracle on NetApp filers? I am looking for information for a project I am involved in.
- How does performance really stack up compared with a DAS or SAN solution? Does the IP stack -vs- SCSI stack cause a significant performance lag?
In our environment, we have typically seen faster writes to the filer than to local disk (Sun UltraSCSI1 10000 rpm). We have GbE fibre between our servers and our filers and we have not seen a problem with performance. We have one F740 running about 8 instances and another two clusters of F760C's running a couple more.
- Are there compelling reasons why this would be a preferred solution?
Backups - set the database in backup mode for about five minutes and that's all. During that time you can create a snapshot of an entire filesystem and start the backup (using NDMP capable backup software).
- Are there compelling reasons why this solution should be avoided?
We did move one database off the filers because it was doing many small reads and writes (about 4 bytes each). Since the typical Oracle write size is 8K, there was a lot of wasted space in each network packet; we broke down and bought a Sun A5000.
- What are the gotchas of such a solution? Are there configuration issues that can trip you up?
We are looking at converting some of our instances from UDP to TCP, but we're not seeing immediate pressure to make the move, so we're going to hold off until it looks like it might be a problem.
I've read the whitepapers on the NetApp and Oracle sites. I am looking for someone though that has actually done this.
Thanks,
Robin winslett@ev1.net
Geoff Hardin geoff.hardin@dalsemi.com Ignorance can be cured by learning, Stupidity usually can't.
-
Brian Long -
Geoff Hardin -
Igor Schein -
Jeff Kennedy -
Michael van Elst -
Robin Winslett