I'm currently fiddling with a project using the ONTAPI (perf stats monitoring sort of thing).
What I'm wanting to do is reproduce the information from 'df' and 'df -s'.
Doing well so far, because the api 'volume-list-info' seems to have most of the information I want. There's just one thing missing - how much of the 'snap reserve' I'm actually using.
I can't seem to find it in either that, or the 'snapshot' counters. I would ideally be able to do this without having to do a per-volume calculation, because one of the things I'm hoping to do is support a lightweight proxy client that 'just' fetches source XML for processing on a server.
Or is there a way I can calculate this from the size/used/available in the volume-list-info?
Thanks, Ed
7mode?
From a quick look I don’t see that in the data returned on volumes. There is a ‘snapshot_volume_info’ call that returns ‘Total bytes that when exhausted will disable us from taking snapshots’ - which I take to mean free space in the snapshot area. The bad news is that it appears you have to call it with a volume name (though I’ve not tested running it without). So you’d have for call for each volume, and then calculate the used based on reserve and free space.
That’s all I’ve found so far…
--rdp
From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Edward Rolison Sent: Thursday, April 23, 2015 8:58 AM To: toasters@teaparty.net Subject: API calls to replicate a 'df'
I'm currently fiddling with a project using the ONTAPI (perf stats monitoring sort of thing).
What I'm wanting to do is reproduce the information from 'df' and 'df -s'.
Doing well so far, because the api 'volume-list-info' seems to have most of the information I want. There's just one thing missing - how much of the 'snap reserve' I'm actually using.
I can't seem to find it in either that, or the 'snapshot' counters. I would ideally be able to do this without having to do a per-volume calculation, because one of the things I'm hoping to do is support a lightweight proxy client that 'just' fetches source XML for processing on a server.
Or is there a way I can calculate this from the size/used/available in the volume-list-info?
Thanks, Ed
Hi,
On 2015-04-23 14:58, Edward Rolison wrote:
There's just one thing missing - how much of the 'snap reserve' I'm actually using.
That is indeed hard to find. When you call 'snapshot-list-info' on the volume, you'll get back an array of snapshot-info objects. The value you want is in the last snapshot's 'cumulative-total' field (in kilobytes, iirc).
Hope this helps!
Oliver
A little, thanks. At least I know where to find it now. I was rather hoping that I wouldn't have to do a two-pass though. I might have to settle for running a df over ssh instead.
The one silver lining is that I don't need this data with a particularly high resolution - 5 minute CPU loads is useful, 5 minute 'df' isn't nearly so much :).
On 23 April 2015 at 14:47, Oliver Brakmann < oliver.brakmann+toasters@posteo.de> wrote:
Hi,
On 2015-04-23 14:58, Edward Rolison wrote:
There's just one thing missing - how much of the 'snap reserve' I'm actually using.
That is indeed hard to find. When you call 'snapshot-list-info' on the volume, you'll get back an array of snapshot-info objects. The value you want is in the last snapshot's 'cumulative-total' field (in kilobytes, iirc).
Hope this helps!
Oliver _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
In 7 mode, that is actually easier to get with SNMP than the API. (Although you do have to deal with the fact that the volume OID's change when you create/delete volumes - existing volumes will be renumbered.)
In 7 mode, snapshot volumes look just like regular volumes (except they will be called /vol/volname/.snapshot) Then you can just query the usual OIDs: .1.3.6.1.4.1.789.1.5.4.1.29 - the total capacity of the file system .1.3.6.1.4.1.789.1.5.4.1.30 - the used capacity
to get the size/space used of the snapshot reserve. (You may have to use different objects if running older OnTap, that does not support the 64 bit counters.)
But be aware that the 'parent' volume will report its size *not* including the snap reserve, but that snapshot usage that has exceeded the snapreserve will count in the used space of the parent volume.
In cluster mode, LogicMonitor uses this call for space monitoring: <volume-get-iter> <desired-attributes> <volume-attributes> <volume-id-attributes> </volume-id-attributes> <volume-inode-attributes> </volume-inode-attributes> <volume-sis-attributes> </volume-sis-attributes> <volume-snapshot-attributes> </volume-snapshot-attributes> <volume-space-attributes> </volume-space-attributes> <volume-state-attributes> </volume-state-attributes> </volume-attributes> </desired-attributes> <query> </query> </volume-get-iter>
In the returned response is, amongst other things we grab: VOLUME-SPACE-ATTRIBUTES.SNAPSHOT-RESERVE-SIZE volume-space-attributes.percentage-snapshot-reserve-used
LMK if you have further questions. Happy to help. (Of course, you could also give LogicMonitor a try, and have this all solved for you, along with all your other monitoring issues...)
On Thu, Apr 23, 2015 at 6:47 AM, Oliver Brakmann < oliver.brakmann+toasters@posteo.de> wrote:
Hi,
On 2015-04-23 14:58, Edward Rolison wrote:
There's just one thing missing - how much of the 'snap reserve' I'm actually using.
That is indeed hard to find. When you call 'snapshot-list-info' on the volume, you'll get back an array of snapshot-info objects. The value you want is in the last snapshot's 'cumulative-total' field (in kilobytes, iirc).
Hope this helps!
Oliver _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Hmm, hadn't even considered doing it via SNMP. Thanks - that's a useful line of investigation. Sounds like the same limitations as df over ssh, which I can live with.
For one reason or another, I've an isolated environment that I can't hook up to my existing monitoring infrastructure, and whilst a DFM install within the environment would be an option, I've found it remarkably useful to have an 'at a glance' MRTG style set of filer perf graphs. rrdtool has holt-winters aberration detection mechanisms, which with a bit of help from R I'm fiddling with getting automatic aberration detection. (I say 'fiddling' rather than 'working' because whilst I've got such a thing up and running, I'm less sure I actually trust it's results).
Grabbing API extracts and forwarding them on for processing periodically is therefore what I'm aiming to do.
On 23 April 2015 at 16:48, Steve Francis sfrancis@logicmonitor.com wrote:
In 7 mode, that is actually easier to get with SNMP than the API. (Although you do have to deal with the fact that the volume OID's change when you create/delete volumes - existing volumes will be renumbered.)
In 7 mode, snapshot volumes look just like regular volumes (except they will be called /vol/volname/.snapshot) Then you can just query the usual OIDs: .1.3.6.1.4.1.789.1.5.4.1.29 - the total capacity of the file system .1.3.6.1.4.1.789.1.5.4.1.30 - the used capacity
to get the size/space used of the snapshot reserve. (You may have to use different objects if running older OnTap, that does not support the 64 bit counters.)
But be aware that the 'parent' volume will report its size *not* including the snap reserve, but that snapshot usage that has exceeded the snapreserve will count in the used space of the parent volume.
In cluster mode, LogicMonitor uses this call for space monitoring:
<volume-get-iter> <desired-attributes> <volume-attributes> <volume-id-attributes> </volume-id-attributes> <volume-inode-attributes> </volume-inode-attributes> <volume-sis-attributes> </volume-sis-attributes> <volume-snapshot-attributes> </volume-snapshot-attributes> <volume-space-attributes> </volume-space-attributes> <volume-state-attributes> </volume-state-attributes> </volume-attributes> </desired-attributes> <query> </query> </volume-get-iter>
In the returned response is, amongst other things we grab: VOLUME-SPACE-ATTRIBUTES.SNAPSHOT-RESERVE-SIZE volume-space-attributes.percentage-snapshot-reserve-used
LMK if you have further questions. Happy to help. (Of course, you could also give LogicMonitor a try, and have this all solved for you, along with all your other monitoring issues...)
On Thu, Apr 23, 2015 at 6:47 AM, Oliver Brakmann < oliver.brakmann+toasters@posteo.de> wrote:
Hi,
On 2015-04-23 14:58, Edward Rolison wrote:
There's just one thing missing - how much of the 'snap reserve' I'm actually using.
That is indeed hard to find. When you call 'snapshot-list-info' on the volume, you'll get back an array of snapshot-info objects. The value you want is in the last snapshot's 'cumulative-total' field (in kilobytes, iirc).
Hope this helps!
Oliver _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
-- *Steve Francis* | Chief Product Officer steve@logicmonitor.com 805 698 0770
http://www.LogicMonitor.com *Cloud-based performance monitoring*
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Thank you to everyone who gave me help and assistance on this one. There's several useful suggestions to workaround the fact that the API would need multiple passes.
The most useful being - SNMP does it.
However I've also managed to dig something out that I thought I'd share - there's an undocumented API (which I'm sure you will all be wise enough to use at your own risk!)
system-cli
Which allows you to execute commands via the API auth mechanism.
<!DOCTYPE netapp SYSTEM "/na_admin/netapp_filer.dtd"><netapp version="1.7" xmlns="http://www.netapp.com/filer/admin"> <system-cli> <args> <arg>df</arg> <arg>-k</arg> </args> </system-cli></netapp>
As an example (and apologies if this format mashes - it's perl, run it through perltidy again and it'll be fine). This _doesn't_ use the NetApp SDK, but rather the two CPAN modules 'LWP' and 'XML::Twig'. Both are readily available, and personally I find them altogether easier to work with. (YMMV of course).
The downside is - you get back a single 'cli-output' element that's plain text formatted, so has all the downsides of 'ssh $host df -k'.
(I don't know if that changes in CDOT, but one of my feature requests would be enabling XML/CSV output from all NetApp commands - I've used another vendor's stuff, and this sort of thing is a real boon)
#!/usr/bin/env perl use strict; use warnings;
use XML::Twig; use LWP;
my $twig = XML::Twig->new( 'pretty_print' => 'indented' ); $twig->set_root( XML::Twig::Elt->new( 'netapp', { version => 1.7, vfiler => "somevfiler", xmlns => "http://www.netapp.com/filer/admin", }, ) ); my $api_req = $twig->root->insert_new_elt('system-cli'); my $args = $api_req->insert_new_elt('args'); $args->insert_new_elt( 'last_child', 'arg', 'df' ); $args->insert_new_elt( 'last_child', 'arg', '-k' );
$twig->set_doctype('netapp SYSTEM "file:/etc/netapp_filer.dtd"'); $twig->set_xml_version("1.0"); $twig->set_encoding('utf-8');
$twig->print;
exit;
my $user_agent = LWP::UserAgent->new( 'ssl_opts' => { 'verify_hostname' => 0, 'SSL_version' => 'SSLv3', } );
my $request = HTTP::Request->new( 'POST' => 'https://myfilername/servlets/netapp.servlets.admin.XMLrequest_filer ' ); $request->authorization_basic( 'username_here', 'password_here' ); $request->content( $twig->sprint );
my $results = $user_agent->request($request); if ( not $results->is_success ) { print "Error: ", $results->status_line; exit; }
my $results_xml = XML::Twig->new( 'pretty_print' => 'indented_a' ); $results_xml->parse( $results->content ); $results_xml->print;
On 23 April 2015 at 13:58, Edward Rolison ed.rolison@gmail.com wrote:
I'm currently fiddling with a project using the ONTAPI (perf stats monitoring sort of thing).
What I'm wanting to do is reproduce the information from 'df' and 'df -s'.
Doing well so far, because the api 'volume-list-info' seems to have most of the information I want. There's just one thing missing - how much of the 'snap reserve' I'm actually using.
I can't seem to find it in either that, or the 'snapshot' counters. I would ideally be able to do this without having to do a per-volume calculation, because one of the things I'm hoping to do is support a lightweight proxy client that 'just' fetches source XML for processing on a server.
Or is there a way I can calculate this from the size/used/available in the volume-list-info?
Thanks, Ed
-
Edward Rolison -
Oliver Brakmann -
Payne, Richard -
Steve Francis