Other items to check:
Do you have overlapping/frequent snap mirror jobs? Are you running dedup jobs? Do you have misaligned IO ? http://www.vmadmin.info/2010/07/quantifying-vmdk-misalignment.html http://www.vmadmin.info/2010/07/quantifying-vmdk-misalignment.html Try deconstructing your IOPS to determine source of latency spikes: http://www.vmadmin.info/2010/07/vmware-and-netapp-deconstructing.html http://www.vmadmin.info/2010/07/vmware-and-netapp-deconstructing.html
On Feb 12, 2015, at 1:58 AM, Edward Rolison ed.rolison@gmail.com wrote:
Thanks for all the answers thus far. They've been helpful.
I'm bottle necked on CPU primarily. I've checked the drives, and there's no particular hotspots there - I have nice big aggregates with ~200 spindles, and they're comfortably less than 30%. Nor network - trunked 10G cards, and none are particularly 'hot'. (Nor anywhere else I'm looking).
But the 'sysstat -x' cpu load is in the mid-high 90s, and the 'sysstat -M' is giving a sum of around 850% (12CPUs).
I do have particular volumes 'running hot' that correlates with latency spikes on another volume (and is getting complaints from another user group). By 'running hot' I mean '10K IOPs' and 200MB/sec read, 200MB/sec write. Whilst it's doing that, if my 'other customer' tries to use their share, they get pretty persistent 20ms+ latency (from 'stats show volume') at about 1000 iops/20MB read/sec.
I'm pretty sure this is my root cause, but my 'gut feeling' is that it shouldn't be. Hence the question.
On 12 February 2015 at 01:11, Klise, Steve <Steve.Klise@wwt.com mailto:Steve.Klise@wwt.com> wrote: I agree with Jeff… Could be a bunch of things.. I know in the past if you added storage and didn’t do a reallocate, could be the cause.. Could be lots of things, but a perfstat is a good 1st pass. Look at performance advisor if you have it.. Maybe you have some hot disks.. A little more info such as “what’s slow” would be appreciated to assist further.
From: toasters-bounces@teaparty.net mailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net mailto:toasters-bounces@teaparty.net] On Behalf Of Jeff Mohler Sent: Wednesday, February 11, 2015 5:00 PM To: Edward Rolison Cc: toasters@teaparty.net mailto:toasters@teaparty.net Subject: Re: IOPs from a controller
I suppose, somewhere between 1 and 100,000 would be about right.
I would call your sales machine, and ask for a PS Performance Review.
Not that Support cant say what its doing, but PS will tell you where it's been...and where it's going.
On Wed, Feb 11, 2015 at 5:54 PM, Edward Rolison <ed.rolison@gmail.com mailto:ed.rolison@gmail.com> wrote:
Does anyone have a very rough rule of thumb as to IOPs you might expect out of a controller?
I've got a FAS6280 that's starting to 'be a bit sluggish'. I'm inclined to shrug and say 'yes, it's doing 20K IOPs'.
I know there's very much an 'it depends' in there - but 'very rough' is good enough for me here. (Disk wise - it's not particularly high on disk utilisation, nor is the network bandwidth particularly full).
Toasters mailing list Toasters@teaparty.net mailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters http://www.teaparty.net/mailman/listinfo/toasters
--
Gustatus Similis Pullus
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters