Fellow Toasters, We purchased a new FAS3050c with the intent of using it for Oracle over NFS. This will be our first exposure to Oracle over NFS as we are migrating from a SAN environment (EMC Symm 8430), so we need to ensure we get as good a performance we got from our SAN. The filers would also be serving regular CIFS and NFS data.
Here's the configuration, please suggest the best way to lay out the aggregates in order to provide optimum performance for Oracle over NFS. Filer Configuration: - FAS3050c with 6 shelves (3 shelves per head). Shelves are with 300GB 10K FC drives. No additional add-on cards, just with 4 built-in Gig-E and FC ports each. - The 3 shelves are all on a single loop now, would this be a problem? There are 2 FC ports free for future shelf expansion. Should we start off with 2 loops now, 1 loop with 2 shelves and 1 with just 1 shelf? - On each head, we have created a VIF using 2 NICs each, exclusive for Oracle NFS traffic with Jumbo frames enabled and hooked to a dedicated Gigabit switch with no routing outside. One NIC is configured for regular CIFS/NFS traffic and will be on our normal LAN. The 4th NIC is un-configured at present. There will be 4 Solaris 10 boxes running Oracle, 2 production and 2 dev, each with 1 NIC dedicated for oracle NFS and jumbo frames enabled. We may add a 2nd NIC to the production hosts and trunk it for Oracle NFS in the future and at that time add the 3rd NIC on the filers to the VIF. Does this network configuration look reasonable. The oracle application will have about 100 to 150 users running ERP, with about 50 to 60 concurrent users. The database size is not large, under 500GB (including dev and production)
Now the question on creating the aggregates. We definitely want to get the best performance possible and at the same time don't want to sacrifice too much capacity by creating smaller RAID groups. So this is what I have in mind:
On each head, create 2 aggregates with 20 disks each, leaving 2 disks as spare. The aggregates will have a raid size of 20 disks (instead of the default 16), there by maximizing the disks for data (about 4 TB usable per aggregate). This means there is just 1 RAID set per aggregate. Is this OK? What would be the impact when there is a disk failure and rebuilds? Is it better to create smaller raid groups with a raidsize of 10 to minimize rebuild performance? Should we look at creating 1 single aggregate with 40 disks? I know more the spindles the better for an aggregate, how big a difference will a 40 disk aggregate make over a 20 disk one? When does the spindle count max out performace?
Hope this is enough information to start a meaningful discussion. Looking forward to all of your valuable inputs. The NetApp white papers don't have these kind of detailed configuration inforamtion.
TIA -G