toasters

toasters@lists.teaparty.net

13513 discussions

na_snapmon: snapshot monitor
by Daniel Quinlan 01 Jul '98

01 Jul '98

I finally got fed up with deleting snapshots by hand when they started using too much disk space, so I created a simple snapshot monitor that automatically deletes snapshots when they start using more than 100% of the space reserved for snapshot usage. Now, I configure each filer to schedule more snapshots than I really want and I use only "snap reserve" to limit how much disk usage is taken by the snapshots. If a filer's data turnover rises, it automatically scales the number of snapshots back. For example: Jun 26 10:30:36 sodium na_snapmon: rsh uranium snap delete hourly.3 Jun 26 10:33:03 sodium na_snapmon: rsh neptunium snap delete nightly.1 Jun 26 10:34:49 sodium na_snapmon: rsh thorium snap delete hourly.2 Jun 26 10:40:04 sodium na_snapmon: rsh neptunium snap delete hourly.3 It will also try to delete dump snapshots that are older than two days old, and it will also ignore snapshots that have been specially created or renamed. The best way to run this is as root cron job, every ten minutes or so. This runs on Linux now, but should also work on Solaris. Other Unix systems with very minimal modification. ------- start of cut text -------------- #!/usr/bin/perl # na_snapmon - monitor snapshot usage and keep it under control # Copyright (C) 1998 Daniel Quinlan # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. use Getopt::Std; require "find.pl"; sub usage; sub trim_snapshots; sub snap_delete; my $limit; my $prog; $prog = $0; $prog =~ s@.*/@@; getopts("hd"); if ($opt_h) { &usage; exit 0; } foreach $dir (@ARGV) { my $df_worked = 0; if (! -d $dir) { warn("$prog: $dir: no such snapshot mount point"); } open (DF, "df -k $dir |"); while (<DF>) { if (/[0-9]%/) { $df_worked = 1; ($fs, $blocks, $used, $avail, $capacity, $mount) = split; if ($capacity !~ /^[0-9]?[0-9]%/) { $fs =~ s/:.*//; trim_snapshots($fs, $dir); } } } close (DF); } sub trim_snapshots { my ($filer, $dir) = @_; my (@list, @snapshots, %atime); # read snapshot list opendir(DIR, $dir) || die "can't opendir $dir: $!"; @snapshots = grep { ! /^\.\.?$/ && -d "$dir/$_" } readdir(DIR); closedir DIR; # sort by atime foreach $entry (@snapshots) { $atime{$entry} = -A "$dir/$entry"; } @snapshots = sort {$atime{$a} <=> $atime{$b}} @snapshots; # maybe delete something while ($s = (pop @snapshots)) { # try to delete old dump snapshots, but since this might # fail, keep trying to delete after this one. if ($s =~ /^snapshot_for_dump\.[0-9]+$/ && time - $atime{$s} >172800) { snap_delete($filer, $s); next; } # skip manual snapshots next if ($s !~ /^(hourly|nightly|weekly)\.[0-9]+$/); # leave one snapshot at all times next if ($s =~ /^hourly\.0$/); snap_delete($filer, $s); last; } } sub snap_delete { my ($filer, $snapshot) = @_; if ($opt_d || ($> != 0)) { print "debug: rsh $filer snap delete $snapshot\n"; } else { system("logger -t na_snapmon \"rsh $filer snap delete $snapshot\""); system("rsh $filer snap delete $snapshot"); } } sub usage { print <<EOF; usage: $prog [-hd] [list of snapshot mount points] -h print this help -d debugging mode: don't do, just show EOF } ------- end ----------------------------

1 0

Quotas
by Raymond Brennan 01 Jul '98

01 Jul '98

I'm sort of new to all this, so here goes : I want to create a root directory called, say, dog. I then want to create a subdirectory called pup. I want to limit the size to which pup can grow to, say, 50 Mb. The thing is, the quota qtree command won't accept slashes in the qtree name. Also, if I create "dog" by doing # mkdir dog then do #cd dog # mkdir pup then go to the quotas file on /etc and do /dog/pup 10G 10K and run quota on I get "/dog/pup is not a quota tree. In short, how do I set quotas on file systems a number of steps down the tree ? Why can't I do /dog/pup 5G /dog/puppy 5G

1 0

OT 5.?
by Weeks, Thomas 01 Jul '98

01 Jul '98

Just curious... how is the OT 5.x software working for eveyone? Any problems? Stability probs? Backout (to 4.3.4) problems? What about quotas and the like? Just curious... Tom > -----Original Message----- > From: Mark D Simmons [SMTP:mds@gbnet.net] > Sent: Monday, June 29, 1998 1:48 AM > To: quinlan(a)transmeta.com; toasters(a)mathworks.com > Subject: Re: what data is being used? > > I recall asking for this in an RFE sense a year ago as it would be > incredibly > useful. Sun just put the capability to track the load on disks at the > partition > level in Solaris 2.6 - this sort of stat on a filer would be just as > useful IMHO > > Per quota tree seemed most natural to me as that is already a > distinctive > structure in filer housekeeping meta-data. Directory level gathering > might be > too much load to put on a filer which is meant to perform under high > demand... > > ...how about an admin command that flips a bit on a directory and > makes it a > watchpoint in the filesystem - accesses against anything beneath it > count on > that directory's stats. Even if the total number of such points were > limited > to put a boundary on overhead in the filer, it would still be a useful > analytical tool.

1 0

Re: what data is being used?
by Mark D Simmons 29 Jun '98

29 Jun '98

I recall asking for this in an RFE sense a year ago as it would be incredibly useful. Sun just put the capability to track the load on disks at the partition level in Solaris 2.6 - this sort of stat on a filer would be just as useful IMHO Per quota tree seemed most natural to me as that is already a distinctive structure in filer housekeeping meta-data. Directory level gathering might be too much load to put on a filer which is meant to perform under high demand... ...how about an admin command that flips a bit on a directory and makes it a watchpoint in the filesystem - accesses against anything beneath it count on that directory's stats. Even if the total number of such points were limited to put a boundary on overhead in the filer, it would still be a useful analytical tool.

1 0

Zowie! He's using 4 TB!
by Brian Rice 26 Jun '98

26 Jun '98

Has anyone else seen this little sparkle in the output of quota report? user sairam 4294947972 5120000 4294967282 512000 This shows up intermittently after I move a user to the filer with a restore rfD - . As you might imagine, it prevents the user from writing. Workaround is quota off; quota on. This is on an F630 running 4.3.3. I haven't filed a bug report yet, though I will as soon as I have confidence that I can state the sequence of events that triggers this phenomenon. Just curious to see if anyone else had seen it. Brian Rice brice(a)gnac.com

2 1

what data is being used?
by Daniel Quinlan 26 Jun '98

26 Jun '98

Are there any tools to gauge NFS load (reads, writes, NFS ops, whatever), perhaps on a quota tree or directory level. Why would I want to know such a thing? Say I have an 200GB F540 filer with two large trees. And I just purchased a 200GB F630 filer. I want to move one of the trees, but not both. I want to move the busier one, not the other one. There are some basic things I've been looking at in the above situation, but maybe someone here knows about a better method/technique/tool. I hoping someone will mention such an obvious solution that I'll be glad I didn't mention what sort of thing I've been doing. :-) - Dan

1 0

Re: Making CIFS hidden files from unix
by Daniel Quinlan 26 Jun '98

26 Jun '98

Steve Losen <scl(a)sasha.acc.virginia.edu> writes: > When we add new user accounts to our NetApp, we don't know if the owner > will use CIFS, unix, or both. (We use unix file protection semantics.) So > we are going to put unix dot files in the new home directory. But it > would be nice to prevent or discourage CIFS users from deleting them -- in > particular the .forward file. We were thinking that if we could make the > dot files "hidden" and "system", that that would help, even though CIFS > users can override and see hidden files if they want. > > [...] > > Is this overly complex? Is there a simpler way? Yes. I think so. A simpler solution is to configure sendmail to look for .forward files somewhere other than home directories. We put all .forward files (as well as procmail stuff) in a centralized location (physically on the mail server). For example, /local/mail/<username>/.forward /local/mail can be (auto)mounted to allow editing on clients. You can put symbolic links in user home directories, and it won't matter if people delete the links. This also allows for faster and more reliable mail delivery since the .forward files are now located on the mail server. You don't need to worry about someone's home directory being unavailable. - Dan

1 0

Making CIFS hidden files from unix
by Steve Losen 25 Jun '98

25 Jun '98

When we add new user accounts to our NetApp, we don't know if the owner will use CIFS, unix, or both. (We use unix file protection semantics.) So we are going to put unix dot files in the new home directory. But it would be nice to prevent or discourage CIFS users from deleting them -- in particular the .forward file. We were thinking that if we could make the dot files "hidden" and "system", that that would help, even though CIFS users can override and see hidden files if they want. Our account adding software is on unix, and I think I have a way to do what I want. I'll make a "new user" prototype directory and set the unix permissions on the dot files, then get on a PC and set "hidden" and "system" on them. I'll use the netapp dump utility to dump the prototype dir to a file. When I create a user, I'll use the netapp restore to bring back the prototype directory, chown the files to the new user, overwrite the .forward, and mv the prototype into place as the user's home dir. This preserves any CIFS hidden attributes, I presume. It looks like restore always restores back to the original location of the dumped directory. Is it possible to get restore to restore elsewhere? We're running NetApp Release 4.3.1. Is this overly complex? Is there a simpler way? Steve Losen scl(a)virginia.edu phone: 804-924-0640 University of Virginia ITC Unix Support

2 1

Re: always safe to delete dump snapshots?
by Paul Albers 25 Jun '98

25 Jun '98

Dan and Bruce, This was bug #650 and has been fixed in all the OnTap OS releases since 4.3x. The request to delete this snapshot_for_dump.<num> will fail. -paul At 08:42 AM 6/25/98 -0500, you wrote: >On 06/25/98 01:38:27 you wrote: >> >>Is it always safe to attempt to delete a snapshot_for_dump.<num>? >> >>Another way to phrase my question -- it will *never* succeed when a >>dump is in progress, right? >> >>Yes, I know it's supposed to not succeed, but since I'm writing a >>program to automatically delete snapshots, I'm exercising more >>paranoia than usual. > >Well, there USED to be an old bug that's been around since the >beginning that trying to delete one while a dump was in progress >would crash the filer. This bug was even mentioned in bold >letters in the admin guide (either in the snapshot or the dump >sections). Assuming they've fixed it to do what you say, then >you're correct. > >If they haven't, then you don't want to automatically delete those >files, unless you have some way of knowing ahead of time that they >are no longer associated with a dump process (say, one that failed >in some ungraceful way). Since this usually happened because of a >crash, one thing I used to do was include an automatic snap delete >at the end of the rc file to delete the snapshot_for_dump after the >filer had started up. The appearance of the snapshot_for_dump.n >mechanism broke that... I made the suggestion that having the OS >automatically delete those might be a good idea, but on the other >hand it might not... the user may really want to restart the >backup off that old image, rather than taking a new one right at >that moment. > >Bruce > > >

1 0

Re: always safe to delete dump snapshots?
by varun＠netapp.com 25 Jun '98

25 Jun '98

> From sirbruce(a)ix.netcom.com Thu Jun 25 06:45:51 1998 ... > Well, there USED to be an old bug that's been around since the > beginning that trying to delete one while a dump was in progress > would crash the filer. This bug was even mentioned in bold > letters in the admin guide (either in the snapshot or the dump > sections). Assuming they've fixed it to do what you say, then > you're correct. > This is bug 650. Fixed since release 4.2. You should get a "Snapshot in use by dump." message if you attempt to delete a snapshot that's being dumped.

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

toasters