On 22/05/12 13:44, Fletcher Cocquyt wrote:

Welcome to the dollars for IOPS game ! Assuming a 20ms response time threshold - 15K RPM SAS can deliver 220 IOPS vs SATAs 44 IOPS (about 5x) So SAS will get you the IOPS your VMs need.

We've got a 256GB flashcache, and you can tell it's working, as the second operation on data is quick.

Here's the report from my performance case:

Perfstat iteration 4 demonstrates the highest NFS latency - read latency is around 20-40ms and write latency around 15-25ms. That is pretty high.

Aggr1 plex0 is the only aggregate showing activity. It is extremely busy reporting about 250 operations per disks, peaking at 507!

That is extremely high activity for7.2K RPM disks.

In iteration 4, 16% of the writes are misaligned. Fixing this will make disk activity more efficient and we should see fewer operations per disk.

I've moved the worst of the misaligned data off to a CIFS volume on the partner.

You should also check out where your IOPs are being spent - are you doing any snap mirror operations? de-dup?

No snapmirror, but OnCommand Host VM backups every two hours, with snapvault overnight. Dedup is run at midnight and saving about 30%.

http://www.vmadmin.info/2010/07/vmware-and-netapp-deconstructing.html

Looking at our top 5 disks by disk_busy, they're sitting above 75% a lot of the time, peaking at 95%. There's 11 disks in the raid group, a background noise of 100 IOPs from Exchange (iSCSI), and 250 from VMware (NFS v3), so that's soaking up most of the 400 available from our disks. During VM snapshotting there's a few thousand.

Also which VMs are hogging the IOPs? You can download a free trial of vCenter Operations and see via the VM IO heat maps which VMs might be moved to local disk on the ESXi hosts (to offload the netapp)

I do have a vCOPs standard license, and it was a bit useful but also a bit hard to understand. I've moved some of our internal VMs with high IO to the other side of the metrocluster. but we have a new SQL application coming into production soon and it must perform well.

good luck - would like to hear how it turns out!

I'm yet to get pricing, but I'm thinking SAS is the way to go.

On 22/05/12 13:44, Fletcher Cocquyt wrote:

Welcome to the dollars for IOPS game ! Assuming a 20ms response time threshold - 15K RPM SAS can deliver 220 IOPS vs SATAs 44 IOPS (about 5x) So SAS will get you the IOPS your VMs need.

It turned out that two DS14mk4 trays of 600GB FC was not much more than two DS14mk2 trays of 2TB SATA. SAS turned out to be way cheaper than 12x600GB 15krpm SAS disks, I think mostly due to the cost of the ATTO 6500N FC-SAS bridges required for a metrocluster. It's not actually installed yet, so I can't give you actual performance comparisons.

Speaking of metrocluster and SyncMirror, I'm going to reassign the old SATA disks to the other head and add them to its SATA aggregate. What's the best way to reallocate volumes that are being SyncMirrored?

Also which VMs are hogging the IOPs? You can download a free trial of vCenter Operations and see via the VM IO heat maps which VMs might be moved to local disk on the ESXi hosts (to offload the netapp)

While waiting for the disk to arrive, we put 6x 256GB OCZ Vertex 4 SSDs into a RAID 5 and put the most important SQL server on that, which is performing adequately ;) Since we don't use SMSQL I'm just using OSSV to backup the native SQL backups.

Having used vCOps for a while, I'm don't think I'd pay retail price for it.