If the non-sequential IO is mostly reads you may want to consider increasing your system's memory before looking at the disks. RAM is even faster than a PAM card & you save yourself the context switching + Oracle's (at least in theory) going to do a better job managing your cache than the filer. We have an extreme case here where we're seeing 95%+ reads from disk! It's completely random (think of a 15 year old database that's never had a DBA or been properly indexed).

Essentially the disk subsystem is acting as RAM, not very efficient consider the DB is quite small and the CPU gets burned up performing context switching. PA< cards have helped us a ton but really we need more system RAM so we can bump the PGA.

From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Steiner, Jeffrey Sent: Monday, September 26, 2011 12:19 PM To: Kießl Walter; Tommy.Fallsen@kongsberg.com; toasters@teaparty.net Subject: RE: Oracle and VMware Datastores

You really need to look at the db_file_sequential_read (random IO) activity. If you have heavy random IO then the latency limitations of SATA disks are quickly reached. The FlashCache (PAM-2) cards are much larger than PAM-1 and help a lot, but there are some databases that are simply too large with too much random IO distributed across the datafiles. In those cases, SAS/FC disks are required. On occasion you can find a database where particular datafiles are getting hit with especially heavy random IO and they're candidates for SSD drives, but most of the time FlashCache will cover those needs at a lower cost. If you have heavy db_file_scattered_read (sequential IO) then SATA disks work well and can almost always supply more IO than a database server is capable of processing.

If you have questions you can always ask your NetApp sales team for assistance. In general, an AWR/statspack report showing the performance requirements is enough to figure out if FC/SAS is required or if SATA will do, and if there will be a benefit from FlashCache.

From: Kießl Walter [mailto:kiessl@heidenhain.de] Sent: Monday, September 26, 2011 3:07 PM To: Tommy.Fallsen@kongsberg.com; toasters@teaparty.net Subject: AW: Oracle and VMware Datastores

Hi,

in addition to the discussion about how to lay out the oracle databases within our without vsphere files I have spotted another thing:

I would strongly recommend to put only oracle binaries and oracle test databases on SATA disks. Do not put any other oracle data on SATA.

We have experienced performance issues with SATA disks on our FAS6070 (with PAM-I card). In my opinion, SATA throughput is not suitable for productive databases -regardless of accessing the oracle database within vmdk-files or within dedicated volumes via NFS/iSCSI/FCP.

Best Regards

i. A. Dipl.-Inform. (FH) Walter J. Kießl

------------------------------------------------------------

mailto:kiessl@heidenhain.de

tel.: +49 8669 31 1954

fax: +49 8669 32 1954

------------------------------------------------------------

DR. JOHANNES HEIDENHAIN GmbH

Dr.-Johannes-Heidenhain-Str. 5

83301 Traunreut, Deutschland

http://www.heidenhain.de/

Von: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] Im Auftrag von Tommy.Fallsen@kongsberg.com Gesendet: Freitag, 23. September 2011 10:50 An: toasters@teaparty.net Betreff: Oracle and VMware Datastores

Hi

Im setting a new Oracle Database solution to replace a old Oracle RAC on EVA8400.

After reading Oracle Databases on VMware vSphere 4 - Essential Deployment Tips i got a few question still needs answered.

Our new storage is a FAS 3240 7-Mode

We have a SATA and SAS aggregate and 1TB Flash cache available.

SAS 1 Raid Group (16 disks)

SATA 4 Raid Group (56 disks)

Whats the recommendations for Datastores and VMDK's?

From i can gather i need to create a volume dedicated to Oracle and present(NFS) it as Datastore for Oracle only.

And create VMDK's like this:

VMDK-1 OS from another Datastore

Oracle Datastore

VMDK-2 Oracle Binaries VMDK-3 Oracle data VMDK-4 Oracle Redo VMDK-5 Oracle Archivelogs

I have 5 databases so 5 VM's where i create a volume with Datastore for each?

Would like input from the list how others have done it.

- Tommy Fallsen

SA/DBA Grunt

________________________________

CONFIDENTIALITY This e-mail and any attachment contain KONGSBERG information which may be proprietary, confidential or subject to export regulations, and is only meant for the intended recipient(s). Any disclosure, copying, distribution or use is prohibited, if not otherwise explicitly agreed with KONGSBERG. If received in error, please delete it immediately from your system and notify the sender properly.

------------------------------------------------------------------------------------------------------ Registergericht: Traunstein / Registry Court: HRB 275 - Sitz / Head Office: Traunreut Aufsichtsratsvorsitzender / Chairman of Supervisory Board: Rainer Burkhard Geschäftsführung / Management Board: Thomas Sesselmann (Vorsitzender / Chairman), Michael Grimm, Matthias Fauser, Sebastian Tondorf

E-Mail Haftungsausschluss / E-Mail Disclaimer http://www.heidenhain.de/disclaimer

Please be advised that this email may contain confidential information. If you are not the intended recipient, please notify us by email by replying to the sender and delete this message. The sender disclaims that the content of this email constitutes an offer to enter into, or the acceptance of, any agreement; provided that the foregoing does not invalidate the binding effect of any digital or other electronic reproduction of a manual signature that is included in any attachment.

Tommy,

I would also recommend going to Direct-NFS: very simple, low overhead, scalable.

I would suggest that you look at the restore scenarios that you would like to consider doing using SnapRestore. SnapRestore is fantastic and I can only recommend having a backup strategy making use of storage snapshot features (think about the time this database having grown or another database of size 5TB would need as hours to be restored!). Most use cases of restore are related to the datafiles (and anyway this is the only part which grows a lot).

But if you do volume level restore (which you are not obliged to do, file level is possible, volume level being easier and faster), you should split what is expected to be restored from what needs to be kept. For single instance, these are typically - binaries and configuration files (listener.ora, spfile, password file, etc.) - datafiles (including undo and temp) - online redolog files - archived redolog files - block change tracking - controlfiles

Among these, even with the very good stability and reliability of FAS systems, it makes sense to multiplex online redolog files and controlfiles on two NFS mounts. And the datafiles should be split from all of the others so that when you snaprestore the volume having the datafiles, you still have the latest copies of control files and most importantly you do not snaprestore your online redolog and archived redolog files (they are needed for the recovery!).

So _minimum_ (better with three copies of the control files and online redolog: they will save you in case of major problem), you need: - volume1: controlfile A, online redolog member A of each redolog group, block change tracking, binaries - volume 2: controlfile B, online redolog member B of each redolog group - volume 3: datafiles I recommend, if you have, to have volume 1 and volume 2 on two different filers and if possible on two different FAS cluster.

With RAC, you need to add the OCR and the voting disks, these should not be on volume 3 for the same "restore use case" reason.

cheers Eric

On 26 September 2011 10:43, Tommy.Fallsen@kongsberg.com wrote:

Thanks for all the input. I was surprised the about having SAS and SATA disk on the same filer can degrade performance.

The idea was to dedicate SAS + Flash cache to oracle DB on filer1 and SAS + Flash cache to MSSQL on filer2

Anyway this network where the filers are located is not performance oriented. Stable and resiliant system was

primary goal.

We have a another network with FAS3270 where we seperated SAS to one filer and SATA to another

Filer for best performance. Vfilers with almost 200 mill. files total is located on the filer with SAS.

Users thought their build system was broken because of the incredible performance difference from the EVA 8400 systems.

The Oracle databases are small. Five in total. Largest is 70GB with a 10GB SGA. I'll use AWR on them and get some performance data.

---

*From:* Page, Jeremy [jeremy.page@gilbarco.com] *Sent:* Monday, September 26, 2011 6:38 PM *To:* Steiner, Jeffrey; Kießl Walter; Fallsen, Tommy; toasters@teaparty.net

*Subject:* RE: Oracle and VMware Datastores

If the non-sequential IO is mostly reads you may want to consider increasing your system’s memory before looking at the disks. RAM is even faster than a PAM card & you save yourself the context switching + Oracle’s (at least in theory) going to do a better job managing your cache than the filer. We have an extreme case here where we’re seeing 95%+ reads *from disk!* It’s completely random (think of a 15 year old database that’s never had a DBA or been properly indexed).

Essentially the disk subsystem is acting as RAM, not very efficient consider the DB is quite small and the CPU gets burned up performing context switching. PA< cards have helped us a ton but really we need more system RAM so we can bump the PGA.

*From:* toasters-bounces@teaparty.net [mailto: toasters-bounces@teaparty.net] *On Behalf Of *Steiner, Jeffrey *Sent:* Monday, September 26, 2011 12:19 PM *To:* Kießl Walter; Tommy.Fallsen@kongsberg.com; toasters@teaparty.net *Subject:* RE: Oracle and VMware Datastores

You really need to look at the db_file_sequential_read (random IO) activity. If you have heavy random IO then the latency limitations of SATA disks are quickly reached. The FlashCache (PAM-2) cards are much larger than PAM-1 and help a lot, but there are some databases that are simply too large with too much random IO distributed across the datafiles. In those cases, SAS/FC disks are required. On occasion you can find a database where particular datafiles are getting hit with especially heavy random IO and they’re candidates for SSD drives, but most of the time FlashCache will cover those needs at a lower cost. If you have heavy db_file_scattered_read (sequential IO) then SATA disks work well and can almost always supply more IO than a database server is capable of processing.

If you have questions you can always ask your NetApp sales team for assistance. In general, an AWR/statspack report showing the performance requirements is enough to figure out if FC/SAS is required or if SATA will do, and if there will be a benefit from FlashCache.

*From:* Kießl Walter [mailto:kiessl@heidenhain.de] *Sent:* Monday, September 26, 2011 3:07 PM *To:* Tommy.Fallsen@kongsberg.com; toasters@teaparty.net *Subject:* AW: Oracle and VMware Datastores

Hi,

in addition to the discussion about how to lay out the oracle databases within our without vsphere files I have spotted another thing:

I would strongly recommend to put only oracle binaries and oracle test databases on SATA disks. Do not put any other oracle data on SATA.

We have experienced performance issues with SATA disks on our FAS6070 (with PAM-I card). In my opinion, SATA throughput is not suitable for productive databases –regardless of accessing the oracle database within vmdk-files or within dedicated volumes via NFS/iSCSI/FCP.

Best Regards

i. A.* Dipl.-Inform. (FH) Walter J. Kießl*

---

mailto:kiessl@heidenhain.de kiessl@heidenhain.de

tel.: +49 8669 31 1954

fax: +49 8669 32 1954

---

DR. JOHANNES HEIDENHAIN GmbH

Dr.-Johannes-Heidenhain-Str. 5

83301 Traunreut, Deutschland

http://www.heidenhain.de/

*Von:* toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] *Im Auftrag von * Tommy.Fallsen@kongsberg.com *Gesendet:* Freitag, 23. September 2011 10:50 *An:* toasters@teaparty.net *Betreff:* Oracle and VMware Datastores

Hi

Im setting a new Oracle Database solution to replace a old Oracle RAC on EVA8400.

After reading Oracle Databases on VMware vSphere 4 - Essential Deployment Tips i got a few question still needs answered.

Our new storage is a FAS 3240 7-Mode

We have a SATA and SAS aggregate and 1TB Flash cache available.

SAS 1 Raid Group (16 disks)

SATA 4 Raid Group (56 disks)

Whats the recommendations for Datastores and VMDK's?

From i can gather i need to create a volume dedicated to Oracle and present(NFS) it as Datastore for Oracle only.

And create VMDK's like this:

VMDK-1 OS from another Datastore

Oracle Datastore

VMDK-2 Oracle Binaries VMDK-3 Oracle data VMDK-4 Oracle Redo VMDK-5 Oracle Archivelogs

I have 5 databases so 5 VM's where i create a volume with Datastore for each?

Would like input from the list how others have done it.

•       Tommy Fallsen
  
              SA/DBA Grunt
  
  
  
  
  

---

CONFIDENTIALITY This e-mail and any attachment contain KONGSBERG information which may be proprietary, confidential or subject to export regulations, and is only meant for the intended recipient(s). Any disclosure, copying, distribution or use is prohibited, if not otherwise explicitly agreed with KONGSBERG. If received in error, please delete it immediately from your system and notify the sender properly.

---

Registergericht: Traunstein / Registry Court: HRB 275 - Sitz / Head Office: Traunreut Aufsichtsratsvorsitzender / Chairman of Supervisory Board: Rainer Burkhard Geschäftsführung / Management Board: Thomas Sesselmann (Vorsitzender / Chairman), Michael Grimm, Matthias Fauser, Sebastian Tondorf

E-Mail Haftungsausschluss / E-Mail Disclaimerhttp://www.heidenhain.de/disclaimer

Please be advised that this email may contain confidential information. If you are not the intended recipient, please notify us by email by replying to the sender and delete this message. The sender disclaims that the content of this email constitutes an offer to enter into, or the acceptance of, any agreement; provided that the foregoing does not invalidate the binding effect of any digital or other electronic reproduction of a manual signature that is included in any attachment.

---

Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters