We have a growing deployment with ~25 filers and about 750TB of usable space. Most of it is VMware datastore space exposed via NFS, but some is general purpose NAS file sharing.
We've been using OnCommand Core (formerly DFM) to handle reporting and such, and for the most part it's worked pretty well.
I need to start pulling together some metrics for the environment as a whole, both to benefit the Storage Team who manages the devices, but also to be able to pass along to my bosses so they can get a basic grasp of how things are going.
I'm curious how those of you with deployments similar to mine or larger hvae dealt with this? What sort of things are you reporting on, and how do you handle the "roll-ups" to summarize many (in our case 200+) volumes into something meaningful? Are you generating reports natively from OnCommand Core, or pulling the data out into another tool? Do you find the OnCommand DataSets feature useful? We haven't really leveraged it properly I think.
My thought is to present a sort of physical view of our environment showing roll-ups of storage capacity and performance (probably CPU, IOPS and latency). How to give a snapshot overview of the whole environment is what I'm not sure on.
I'm thinking using the DataSets will help me organize our various types of data and then I can report on that.
I'm not sure, however, if I should spend my time trying to get the right reports out of OnCommand Core directly or if I should just start extracting the data to another tool (which?) or into a database where I can home-brew up what I'm after.
Thoughts/experiences?
Thanks, Ray
Hi Ray
I have some homebrew scripts which pull data out of my filers via snmp and the cli stats command. These push data to graphite, which allows me do do some interesting queries.
The scripts are pulling basic info, and I plan to add more, but attached are some sample of the detail I can pull
I only have 4 filers and 50TB of disk, but graphite should be able to scale to the size you need. I am currently updating about 40000 metrics per minute on a single graphite backend, and have no need to scale out yet. Those metrics include the netapp stats, and a lot of other linux hosts.
Chris
On 30/07/2013 08:11, Ray Van Dolson wrote:
We have a growing deployment with ~25 filers and about 750TB of usable space. Most of it is VMware datastore space exposed via NFS, but some is general purpose NAS file sharing.
We've been using OnCommand Core (formerly DFM) to handle reporting and such, and for the most part it's worked pretty well.
I need to start pulling together some metrics for the environment as a whole, both to benefit the Storage Team who manages the devices, but also to be able to pass along to my bosses so they can get a basic grasp of how things are going.
I'm curious how those of you with deployments similar to mine or larger hvae dealt with this? What sort of things are you reporting on, and how do you handle the "roll-ups" to summarize many (in our case 200+) volumes into something meaningful? Are you generating reports natively from OnCommand Core, or pulling the data out into another tool? Do you find the OnCommand DataSets feature useful? We haven't really leveraged it properly I think.
My thought is to present a sort of physical view of our environment showing roll-ups of storage capacity and performance (probably CPU, IOPS and latency). How to give a snapshot overview of the whole environment is what I'm not sure on.
I'm thinking using the DataSets will help me organize our various types of data and then I can report on that.
I'm not sure, however, if I should spend my time trying to get the right reports out of OnCommand Core directly or if I should just start extracting the data to another tool (which?) or into a database where I can home-brew up what I'm after.
Thoughts/experiences?
Thanks, Ray _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
I know it has a cost but I can’t pimp LogicMonitor enough, you’d get all of this without hand rolling and the hours involved with doing it.
I get nothing from them, I just think their product frees up hours for other important items, like not burning hours gathering metrics that can be done faster and easier via other methods.
I wrote custom perl scripts that pull the output of "dfm perf data retrieve," puts it into an array, then uses GD::Graph to plot custom graphs. Graphs are created nightly and can be viewed by navigating to a URL on that server. We graph on CPU utilization, FCP latency, CIFS latency, and overall storage capacity. It's works very well and provides a great high level view of our NetApp storage environment.
Regards,
Daniel
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Ray Van Dolson Sent: Tuesday, July 30, 2013 1:11 AM To: toasters@teaparty.net Subject: Metrics!
We have a growing deployment with ~25 filers and about 750TB of usable space. Most of it is VMware datastore space exposed via NFS, but some is general purpose NAS file sharing.
We've been using OnCommand Core (formerly DFM) to handle reporting and such, and for the most part it's worked pretty well.
I need to start pulling together some metrics for the environment as a whole, both to benefit the Storage Team who manages the devices, but also to be able to pass along to my bosses so they can get a basic grasp of how things are going.
I'm curious how those of you with deployments similar to mine or larger hvae dealt with this? What sort of things are you reporting on, and how do you handle the "roll-ups" to summarize many (in our case 200+) volumes into something meaningful? Are you generating reports natively from OnCommand Core, or pulling the data out into another tool? Do you find the OnCommand DataSets feature useful? We haven't really leveraged it properly I think.
My thought is to present a sort of physical view of our environment showing roll-ups of storage capacity and performance (probably CPU, IOPS and latency). How to give a snapshot overview of the whole environment is what I'm not sure on.
I'm thinking using the DataSets will help me organize our various types of data and then I can report on that.
I'm not sure, however, if I should spend my time trying to get the right reports out of OnCommand Core directly or if I should just start extracting the data to another tool (which?) or into a database where I can home-brew up what I'm after.
Thoughts/experiences?
Thanks, Ray _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
This email transmission and any documents, files or previous email messages attached to it may contain information that is confidential or legally privileged. If you are not the intended recipient or a person responsible for delivering this transmission to the intended recipient, you are hereby notified that you must not read this transmission and that any disclosure, copying, printing, distribution or use of this transmission is strictly prohibited. If you have received this transmission in error, please immediately notify the sender by telephone or return email and delete the original transmission and its attachments without reading or saving in any manner.
I've received several message for the scripts, but unfortunately I will not be able to share them in their entirety. They really are custom to my environment, but they shouldn't be too hard to create if you know perl.
foreach $controller (@controllers)
{
.....
@output =`dfm perf data retrieve -o $controller -C system:avg_processor_busy -d $duration`;
.....
}
foreach $day(@day)
{
.....
$data->add_point($data_day,$ctrla_values[$array_counter],$ctrlb_values[$ array_counter],50,90);
.....
}
print IMG $gd->png;
The result would be something like:
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Daniel Keisling Sent: Tuesday, July 30, 2013 8:19 AM To: Ray Van Dolson; toasters@teaparty.net Subject: RE: Metrics!
I wrote custom perl scripts that pull the output of "dfm perf data retrieve," puts it into an array, then uses GD::Graph to plot custom graphs. Graphs are created nightly and can be viewed by navigating to a URL on that server. We graph on CPU utilization, FCP latency, CIFS latency, and overall storage capacity. It's works very well and provides a great high level view of our NetApp storage environment.
Regards,
Daniel
-----Original Message-----
From: toasters-bounces@teaparty.net mailto:toasters-bounces@teaparty.net
[mailto:toasters-bounces@teaparty.net mailto:toasters-bounces@teaparty.net ] On Behalf Of Ray Van Dolson
Sent: Tuesday, July 30, 2013 1:11 AM
To: toasters@teaparty.net mailto:toasters@teaparty.net
Subject: Metrics!
We have a growing deployment with ~25 filers and about 750TB of usable space. Most of it is VMware datastore space exposed via NFS, but some is general purpose NAS file sharing.
We've been using OnCommand Core (formerly DFM) to handle reporting and such, and for the most part it's worked pretty well.
I need to start pulling together some metrics for the environment as a whole, both to benefit the Storage Team who manages the devices, but also to be able to pass along to my bosses so they can get a basic grasp of how things are going.
I'm curious how those of you with deployments similar to mine or larger hvae dealt with this? What sort of things are you reporting on, and how do you handle the "roll-ups" to summarize many (in our case 200+) volumes into something meaningful? Are you generating reports natively from OnCommand Core, or pulling the data out into another tool? Do you find the OnCommand DataSets feature useful? We haven't really leveraged it properly I think.
My thought is to present a sort of physical view of our environment showing roll-ups of storage capacity and performance (probably CPU, IOPS and latency). How to give a snapshot overview of the whole environment is what I'm not sure on.
I'm thinking using the DataSets will help me organize our various types of data and then I can report on that.
I'm not sure, however, if I should spend my time trying to get the right reports out of OnCommand Core directly or if I should just start extracting the data to another tool (which?) or into a database where I can home-brew up what I'm after.
Thoughts/experiences?
Thanks,
Ray
_______________________________________________
Toasters mailing list
Toasters@teaparty.net mailto:Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters http://www.teaparty.net/mailman/listinfo/toasters
This email transmission and any documents, files or previous email messages attached to it may contain information that is confidential or legally privileged.
If you are not the intended recipient or a person responsible for delivering this transmission to the intended recipient, you are hereby notified that you must not read this transmission and that any disclosure, copying, printing, distribution or use of this transmission is strictly prohibited.
If you have received this transmission in error, please immediately notify the sender by telephone or return email and delete the original transmission and its attachments without reading or saving in any manner.
_______________________________________________
Toasters mailing list
Toasters@teaparty.net mailto:Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters http://www.teaparty.net/mailman/listinfo/toasters
This email transmission and any documents, files or previous email messages attached to it may contain information that is confidential or legally privileged. If you are not the intended recipient or a person responsible for delivering this transmission to the intended recipient, you are hereby notified that you must not read this transmission and that any disclosure, copying, printing, distribution or use of this transmission is strictly prohibited. If you have received this transmission in error, please immediately notify the sender by telephone or return email and delete the original transmission and its attachments without reading or saving in any manner.
There is this very simple script that generates CSV to your email with your aggr usage. Requires passwordless ssh from the system/user you cron this on.
It is not much, but figured I would share
#!/bin/bash date=$(date -I) dir=/root/sanreport/
for i in {1..8}; do echo "============= filer$i ===============";ssh filer$i df -gA|grep -v snap|grep -v Aggregate|sort|sed s/GB//g|awk '{print $1,"," $2,"," $3,"," $4}';done > $dir$date-San_Report.csv mutt -s "$date San Capacity Report" -a $dir$date-San_Report.csv emailaddress@domin < $dir/body rm $dir$date-San_Report.csv
From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Daniel Keisling Sent: Tuesday, July 30, 2013 11:53 AM To: Ray Van Dolson; toasters@teaparty.net Subject: RE: Metrics!
I've received several message for the scripts, but unfortunately I will not be able to share them in their entirety. They really are custom to my environment, but they shouldn't be too hard to create if you know perl.
foreach $controller (@controllers)
{
.....
@output =`dfm perf data retrieve -o $controller -C system:avg_processor_busy -d $duration`;
.....
}
foreach $day(@day)
{
.....
$data->add_point($data_day,$ctrla_values[$array_counter],$ctrlb_values[$array_counter],50,90);
.....
}
print IMG $gd->png;
The result would be something like:
[http://storage.wilm.ppdi.com/storage/graphs/AUSSTORE2-CPU-30-line.png]
-----Original Message----- From: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Daniel Keisling Sent: Tuesday, July 30, 2013 8:19 AM To: Ray Van Dolson; toasters@teaparty.netmailto:toasters@teaparty.net Subject: RE: Metrics!
I wrote custom perl scripts that pull the output of "dfm perf data retrieve," puts it into an array, then uses GD::Graph to plot custom graphs. Graphs are created nightly and can be viewed by navigating to a URL on that server. We graph on CPU utilization, FCP latency, CIFS latency, and overall storage capacity. It's works very well and provides a great high level view of our NetApp storage environment.
Regards,
Daniel
-----Original Message-----
From: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net
[mailto:toasters-bounces@teaparty.net] On Behalf Of Ray Van Dolson
Sent: Tuesday, July 30, 2013 1:11 AM
To: toasters@teaparty.netmailto:toasters@teaparty.net
Subject: Metrics!
We have a growing deployment with ~25 filers and about 750TB of usable space. Most of it is VMware datastore space exposed via NFS, but some is general purpose NAS file sharing.
We've been using OnCommand Core (formerly DFM) to handle reporting and such, and for the most part it's worked pretty well.
I need to start pulling together some metrics for the environment as a whole, both to benefit the Storage Team who manages the devices, but also to be able to pass along to my bosses so they can get a basic grasp of how things are going.
I'm curious how those of you with deployments similar to mine or larger hvae dealt with this? What sort of things are you reporting on, and how do you handle the "roll-ups" to summarize many (in our case 200+) volumes into something meaningful? Are you generating reports natively from OnCommand Core, or pulling the data out into another tool? Do you find the OnCommand DataSets feature useful? We haven't really leveraged it properly I think.
My thought is to present a sort of physical view of our environment showing roll-ups of storage capacity and performance (probably CPU, IOPS and latency). How to give a snapshot overview of the whole environment is what I'm not sure on.
I'm thinking using the DataSets will help me organize our various types of data and then I can report on that.
I'm not sure, however, if I should spend my time trying to get the right reports out of OnCommand Core directly or if I should just start extracting the data to another tool (which?) or into a database where I can home-brew up what I'm after.
Thoughts/experiences?
Thanks,
Ray
_______________________________________________
Toasters mailing list
Toasters@teaparty.netmailto:Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
This email transmission and any documents, files or previous email messages attached to it may contain information that is confidential or legally privileged.
If you are not the intended recipient or a person responsible for delivering this transmission to the intended recipient, you are hereby notified that you must not read this transmission and that any disclosure, copying, printing, distribution or use of this transmission is strictly prohibited.
If you have received this transmission in error, please immediately notify the sender by telephone or return email and delete the original transmission and its attachments without reading or saving in any manner.
_______________________________________________
Toasters mailing list
Toasters@teaparty.netmailto:Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
This email transmission and any documents, files or previous email messages attached to it may contain information that is confidential or legally privileged. If you are not the intended recipient or a person responsible for delivering this transmission to the intended recipient, you are hereby notified that you must not read this transmission and that any disclosure, copying, printing, distribution or use of this transmission is strictly prohibited. If you have received this transmission in error, please immediately notify the sender by telephone or return email and delete the original transmission and its attachments without reading or saving in any manner.
I’m just a customer, but LogicMonitor does all of this for you, and a lot more.
www.logicmonitor.com
You can even tell them I sent you, but I don’t get anything anyway.
And for those LM folks reading this: thanks for everything you have put into the NetApp datasource.
I'm not going to spread any rumors here, but the new DFM is supposed to be pretty slick. No idea on ETA, other than its in BETA.. I am keeping my fingers crossed the reporting and "management" dashboard is a bit better.
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Mike Horwath Sent: Wednesday, July 31, 2013 3:14 PM To: Jordan Slingerland Cc: toasters@teaparty.net Subject: Re: Metrics!
I'm just a customer, but LogicMonitor does all of this for you, and a lot more.
www.logicmonitor.com
You can even tell them I sent you, but I don't get anything anyway.
And for those LM folks reading this: thanks for everything you have put into the NetApp datasource. _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
On 01/08/13 06:19, Klise, Steve wrote:
I'm not going to spread any rumors here, but the new DFM is supposed to be pretty slick. No idea on ETA, other than its in BETA.. I am keeping my fingers crossed the reporting and "management" dashboard is a bit better.
OnCommand Unified Manager 6.0 just reached RC, but it's only for Clustered Data ONTAP?
https://communities.netapp.com/community/netapp-blogs/clouds_oncommand/blog/...
Yes and seems somewhat limited. Its in ovf format as well, so I guess VMware only for now. It integrates with WFA so I am guessing some slick new stuff is on the horizon.
________________________________________ From: toasters-bounces@teaparty.net [toasters-bounces@teaparty.net] On Behalf Of James Andrewartha [jandrewartha@ccgs.wa.edu.au] Sent: Friday, August 02, 2013 12:38 AM To: toasters@teaparty.net Subject: Re: Metrics!
On 01/08/13 06:19, Klise, Steve wrote:
I'm not going to spread any rumors here, but the new DFM is supposed to be pretty slick. No idea on ETA, other than its in BETA.. I am keeping my fingers crossed the reporting and "management" dashboard is a bit better.
OnCommand Unified Manager 6.0 just reached RC, but it's only for Clustered Data ONTAP?
https://communities.netapp.com/community/netapp-blogs/clouds_oncommand/blog/...
-- James Andrewartha Network & Projects Engineer Christ Church Grammar School Claremont, Western Australia Ph. (08) 9442 1757 Mob. 0424 160 877 _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
I am doing this by pulling the stats you mentioned (IO, CPU, latency) via SNMP with Icinga (a Nagios fork) and then graphing it with pnp4nagios. Pnp4nagios has a way to define reports with the existing data so I can do stuff like aggregate all the volume's IOPS into a per-filer view and then aggregate that into a cluster view etc.
Volume level example (not sure the list serv will permit images?)
On 07/30/2013 02:11 AM, Ray Van Dolson wrote:
We have a growing deployment with ~25 filers and about 750TB of usable space. Most of it is VMware datastore space exposed via NFS, but some is general purpose NAS file sharing.
We've been using OnCommand Core (formerly DFM) to handle reporting and such, and for the most part it's worked pretty well.
I need to start pulling together some metrics for the environment as a whole, both to benefit the Storage Team who manages the devices, but also to be able to pass along to my bosses so they can get a basic grasp of how things are going.
I'm curious how those of you with deployments similar to mine or larger hvae dealt with this? What sort of things are you reporting on, and how do you handle the "roll-ups" to summarize many (in our case 200+) volumes into something meaningful? Are you generating reports natively from OnCommand Core, or pulling the data out into another tool? Do you find the OnCommand DataSets feature useful? We haven't really leveraged it properly I think.
My thought is to present a sort of physical view of our environment showing roll-ups of storage capacity and performance (probably CPU, IOPS and latency). How to give a snapshot overview of the whole environment is what I'm not sure on.
I'm thinking using the DataSets will help me organize our various types of data and then I can report on that.
I'm not sure, however, if I should spend my time trying to get the right reports out of OnCommand Core directly or if I should just start extracting the data to another tool (which?) or into a database where I can home-brew up what I'm after.
Thoughts/experiences?
Thanks, Ray _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Please be advised that this email may contain confidential information. If you are not the intended recipient, please notify us by email by replying to the sender and delete this message. The sender disclaims that the content of this email constitutes an offer to enter into, or the acceptance of, any agreement; provided that the foregoing does not invalidate the binding effect of any digital or other electronic reproduction of a manual signature that is included in any attachment.
No exactly what you are looking for, but since we are posting pretty graphs.
[cid:image001.png@01CE8D08.81E6BB80] [cid:image002.png@01CE8D08.81E6BB80]
Also, maybe it's just me, but reporting on all the volumes on all those units may be a bit much for an executive summary. Maybe consider just reporting on the aggregates and providing a single %full for each unit. (or possibly per Tier if your units have multiple disk types.
--JMS
From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Jeremy Page Sent: Tuesday, July 30, 2013 9:24 AM To: toasters@teaparty.net Subject: Re: Metrics!
I am doing this by pulling the stats you mentioned (IO, CPU, latency) via SNMP with Icinga (a Nagios fork) and then graphing it with pnp4nagios. Pnp4nagios has a way to define reports with the existing data so I can do stuff like aggregate all the volume's IOPS into a per-filer view and then aggregate that into a cluster view etc.
Volume level example (not sure the list serv will permit images?) [cid:image003.png@01CE8D08.81E6BB80]
On 07/30/2013 02:11 AM, Ray Van Dolson wrote:
We have a growing deployment with ~25 filers and about 750TB of usable
space. Most of it is VMware datastore space exposed via NFS, but some
is general purpose NAS file sharing.
We've been using OnCommand Core (formerly DFM) to handle reporting and
such, and for the most part it's worked pretty well.
I need to start pulling together some metrics for the environment as a
whole, both to benefit the Storage Team who manages the devices, but
also to be able to pass along to my bosses so they can get a basic
grasp of how things are going.
I'm curious how those of you with deployments similar to mine or larger
hvae dealt with this? What sort of things are you reporting on, and
how do you handle the "roll-ups" to summarize many (in our case 200+)
volumes into something meaningful? Are you generating reports natively
from OnCommand Core, or pulling the data out into another tool? Do you
find the OnCommand DataSets feature useful? We haven't really
leveraged it properly I think.
My thought is to present a sort of physical view of our environment
showing roll-ups of storage capacity and performance (probably CPU,
IOPS and latency). How to give a snapshot overview of the whole
environment is what I'm not sure on.
I'm thinking using the DataSets will help me organize our various types
of data and then I can report on that.
I'm not sure, however, if I should spend my time trying to get the
right reports out of OnCommand Core directly or if I should just start
extracting the data to another tool (which?) or into a database where I
can home-brew up what I'm after.
Thoughts/experiences?
Thanks,
Ray
_______________________________________________
Toasters mailing list
Toasters@teaparty.netmailto:Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
Please be advised that this email may contain confidential information. If you are not the intended recipient, please notify us by email by replying to the sender and delete this message. The sender disclaims that the content of this email constitutes an offer to enter into, or the acceptance of, any agreement; provided that the foregoing does not invalidate the binding effect of any digital or other electronic reproduction of a manual signature that is included in any attachment.
Our environment isn't as large as yours, but I wrote a few powershell scripts that pull metrics and push them into graphite and mssql. Things needed for billing reconciliation go into sql and performance/trending into graphite. Using graphite's render API it is fairly straight forward to roll-up like things into a single graph - like sum all IO on each volume on a specific aggregate and divide by the sum of the used space to get the IO density.
mark
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Ray Van Dolson Sent: Tuesday, July 30, 2013 1:11 AM To: toasters@teaparty.net Subject: Metrics!
We have a growing deployment with ~25 filers and about 750TB of usable space. Most of it is VMware datastore space exposed via NFS, but some is general purpose NAS file sharing.
We've been using OnCommand Core (formerly DFM) to handle reporting and such, and for the most part it's worked pretty well.
I need to start pulling together some metrics for the environment as a whole, both to benefit the Storage Team who manages the devices, but also to be able to pass along to my bosses so they can get a basic grasp of how things are going.
I'm curious how those of you with deployments similar to mine or larger hvae dealt with this? What sort of things are you reporting on, and how do you handle the "roll-ups" to summarize many (in our case 200+) volumes into something meaningful? Are you generating reports natively from OnCommand Core, or pulling the data out into another tool? Do you find the OnCommand DataSets feature useful? We haven't really leveraged it properly I think.
My thought is to present a sort of physical view of our environment showing roll-ups of storage capacity and performance (probably CPU, IOPS and latency). How to give a snapshot overview of the whole environment is what I'm not sure on.
I'm thinking using the DataSets will help me organize our various types of data and then I can report on that.
I'm not sure, however, if I should spend my time trying to get the right reports out of OnCommand Core directly or if I should just start extracting the data to another tool (which?) or into a database where I can home-brew up what I'm after.
Thoughts/experiences?
Thanks, Ray _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Hmm... really like this idea. Thanks, Mark.
Ray
On Thu, Aug 01, 2013 at 08:00:23PM +0000, Weber, Mark A wrote:
Our environment isn't as large as yours, but I wrote a few powershell scripts that pull metrics and push them into graphite and mssql. Things needed for billing reconciliation go into sql and performance/trending into graphite. Using graphite's render API it is fairly straight forward to roll-up like things into a single graph
- like sum all IO on each volume on a specific aggregate and divide
by the sum of the used space to get the IO density.
mark
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Ray Van Dolson Sent: Tuesday, July 30, 2013 1:11 AM To: toasters@teaparty.net Subject: Metrics!
We have a growing deployment with ~25 filers and about 750TB of usable space. Most of it is VMware datastore space exposed via NFS, but some is general purpose NAS file sharing.
We've been using OnCommand Core (formerly DFM) to handle reporting and such, and for the most part it's worked pretty well.
I need to start pulling together some metrics for the environment as a whole, both to benefit the Storage Team who manages the devices, but also to be able to pass along to my bosses so they can get a basic grasp of how things are going.
I'm curious how those of you with deployments similar to mine or larger hvae dealt with this? What sort of things are you reporting on, and how do you handle the "roll-ups" to summarize many (in our case 200+) volumes into something meaningful? Are you generating reports natively from OnCommand Core, or pulling the data out into another tool? Do you find the OnCommand DataSets feature useful? We haven't really leveraged it properly I think.
My thought is to present a sort of physical view of our environment showing roll-ups of storage capacity and performance (probably CPU, IOPS and latency). How to give a snapshot overview of the whole environment is what I'm not sure on.
I'm thinking using the DataSets will help me organize our various types of data and then I can report on that.
I'm not sure, however, if I should spend my time trying to get the right reports out of OnCommand Core directly or if I should just start extracting the data to another tool (which?) or into a database where I can home-brew up what I'm after.
Thoughts/experiences?
Thanks, Ray
I've been meaning to do it for a while now, so I put the scripts out on github: https://github.com/mrkwbr/neat/
they aren't the prettiest scripts in the world, but maybe if a few others find them useful, they can help me improve them.
mark
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Ray Van Dolson Sent: Thursday, August 1, 2013 5:33 PM To: toasters@teaparty.net Subject: Re: Metrics!
Hmm... really like this idea. Thanks, Mark.
Ray
On Thu, Aug 01, 2013 at 08:00:23PM +0000, Weber, Mark A wrote:
Our environment isn't as large as yours, but I wrote a few powershell scripts that pull metrics and push them into graphite and mssql. Things needed for billing reconciliation go into sql and performance/trending into graphite. Using graphite's render API it is fairly straight forward to roll-up like things into a single graph
- like sum all IO on each volume on a specific aggregate and divide by
the sum of the used space to get the IO density.
mark
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Ray Van Dolson Sent: Tuesday, July 30, 2013 1:11 AM To: toasters@teaparty.net Subject: Metrics!
We have a growing deployment with ~25 filers and about 750TB of usable space. Most of it is VMware datastore space exposed via NFS, but some is general purpose NAS file sharing.
We've been using OnCommand Core (formerly DFM) to handle reporting and such, and for the most part it's worked pretty well.
I need to start pulling together some metrics for the environment as a whole, both to benefit the Storage Team who manages the devices, but also to be able to pass along to my bosses so they can get a basic grasp of how things are going.
I'm curious how those of you with deployments similar to mine or larger hvae dealt with this? What sort of things are you reporting on, and how do you handle the "roll-ups" to summarize many (in our case 200+) volumes into something meaningful? Are you generating reports natively from OnCommand Core, or pulling the data out into another tool? Do you find the OnCommand DataSets feature useful? We haven't really leveraged it properly I think.
My thought is to present a sort of physical view of our environment showing roll-ups of storage capacity and performance (probably CPU, IOPS and latency). How to give a snapshot overview of the whole environment is what I'm not sure on.
I'm thinking using the DataSets will help me organize our various types of data and then I can report on that.
I'm not sure, however, if I should spend my time trying to get the right reports out of OnCommand Core directly or if I should just start extracting the data to another tool (which?) or into a database where I can home-brew up what I'm after.
Thoughts/experiences?
Thanks, Ray
_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
I had sent a previous mail, but it was too large for the list.
I am doing similar to Mark - I have a few perl scripts pulling stats via snmp and cli perf commands. These get pushed into graphite, and I am using graphitus as a dashboard to view overall stats.
Graphite is fairly powerful - it allows queries like "top 5 average write response times" over all disks to be easily identified.
Chris
On 2013/08/01 10:00 PM, Weber, Mark A wrote:
Our environment isn't as large as yours, but I wrote a few powershell scripts that pull metrics and push them into graphite and mssql. Things needed for billing reconciliation go into sql and performance/trending into graphite. Using graphite's render API it is fairly straight forward to roll-up like things into a single graph - like sum all IO on each volume on a specific aggregate and divide by the sum of the used space to get the IO density.
mark
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Ray Van Dolson Sent: Tuesday, July 30, 2013 1:11 AM To: toasters@teaparty.net Subject: Metrics!
We have a growing deployment with ~25 filers and about 750TB of usable space. Most of it is VMware datastore space exposed via NFS, but some is general purpose NAS file sharing.
We've been using OnCommand Core (formerly DFM) to handle reporting and such, and for the most part it's worked pretty well.
I need to start pulling together some metrics for the environment as a whole, both to benefit the Storage Team who manages the devices, but also to be able to pass along to my bosses so they can get a basic grasp of how things are going.
I'm curious how those of you with deployments similar to mine or larger hvae dealt with this? What sort of things are you reporting on, and how do you handle the "roll-ups" to summarize many (in our case 200+) volumes into something meaningful? Are you generating reports natively from OnCommand Core, or pulling the data out into another tool? Do you find the OnCommand DataSets feature useful? We haven't really leveraged it properly I think.
My thought is to present a sort of physical view of our environment showing roll-ups of storage capacity and performance (probably CPU, IOPS and latency). How to give a snapshot overview of the whole environment is what I'm not sure on.
I'm thinking using the DataSets will help me organize our various types of data and then I can report on that.
I'm not sure, however, if I should spend my time trying to get the right reports out of OnCommand Core directly or if I should just start extracting the data to another tool (which?) or into a database where I can home-brew up what I'm after.
Thoughts/experiences?
Thanks, Ray _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
-
Chris Picton -
Daniel Keisling -
James Andrewartha -
Jeremy Page -
Jordan Slingerland -
Klise, Steve -
Mike Horwath -
Ray Van Dolson -
Weber, Mark A