If the non-sequential IO is mostly reads you may want to consider increasing your system’s memory before looking at the disks. RAM is even faster than a PAM card & you save yourself the context switching + Oracle’s (at least in theory) going to do a better job managing your cache than the filer. We have an extreme case here where we’re seeing 95%+ reads from disk! It’s completely random (think of a 15 year old database that’s never had a DBA or been properly indexed).

Essentially the disk subsystem is acting as RAM, not very efficient consider the DB is quite small and the CPU gets burned up performing context switching. PA< cards have helped us a ton but really we need more system RAM so we can bump the PGA.

You really need to look at the db_file_sequential_read (random IO) activity. If you have heavy random IO then the latency limitations of SATA disks are quickly reached. The FlashCache (PAM-2) cards are much larger than PAM-1 and help a lot, but there are some databases that are simply too large with too much random IO distributed across the datafiles. In those cases, SAS/FC disks are required. On occasion you can find a database where particular datafiles are getting hit with especially heavy random IO and they’re candidates for SSD drives, but most of the time FlashCache will cover those needs at a lower cost. If you have heavy db_file_scattered_read (sequential IO) then SATA disks work well and can almost always supply more IO than a database server is capable of processing.

If you have questions you can always ask your NetApp sales team for assistance. In general, an AWR/statspack report showing the performance requirements is enough to figure out if FC/SAS is required or if SATA will do, and if there will be a benefit from FlashCache.