Just wanted to followup - if you are running <=8.1.2 with dedup your aggregates may be growing due to metadata from sis.
run a sis status -l /vol/<volname> check the status
Last Operation State: Failure Last Successful Operation Begin: Sun Jun 16 23:00:00 PDT 2013 Last Successful Operation End: Sun Jun 23 01:35:09 PDT 2013 Last Operation Begin: Thu Apr 3 10:52:07 PDT 2014 Last Operation End: Thu Apr 3 11:13:54 PDT 2014
and run sis start -s /vol/<volname> to reclaim the space
You get the aggrgate space back and as a side effect you might notice the dedup jobs that had been failing are now returning 20% savings again
Better monitoring/alerting about failed dedup jobs would also prevent this
thanks
On Apr 2, 2014, at 3:47 PM, Fletcher Cocquyt fcocquyt@stanford.edu wrote:
Can confirm
na02> sis start -s /vol/vm65net The file system will be scanned to process existing data in /vol/vm65net. This operation may initialize related existing metafiles. Are you sure you want to proceed (y/n)? y
has brought the aggregate from 90% (18/19Tb) to 82% (16/19Tb) in the last 90 minutes (and space is still being freed)
Pretty ironic a feature (de-dup) designed to save space almost ate all of it! (we have been on 8.1.2 for over a year, and had the same sis config running fine up until this week)
Thanks again
On Apr 2, 2014, at 11:11 AM, Jordan Slingerland Jordan.Slingerland@independenthealth.com wrote:
I think if you dedup from the beginning it should ditch all the old metadata and rebuild the fingerprint database, hopefully you won’t run into the bug again.
sis start –s /vol/vm_volume
From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Fletcher Cocquyt Sent: Wednesday, April 02, 2014 2:01 PM To: Tim Stiller Cc: toasters@teaparty.net Lists Subject: Re: Determining what's contributing to fast aggregate growth
The aggregate growth has stopped since I stopped sis on the vm65net volume. Thanks Tim, and all who replied - this bug was going to fill up our aggregate with 100's of VMs running otherwise.
Now I get to read up more on it and try to reclaim the space Strange this would happen after being stable for so long running de-dup without issue
thanks again, Fletcher
On Apr 2, 2014, at 9:36 AM, Fletcher Cocquyt fcocquyt@stanford.edu wrote:
Yes sis is active
this may be it
na02> sis status -l
Path: /vol/vm65net State: Enabled Compression: Disabled Inline Compression: Disabled Status: Active Progress: 0 KB (0%) Done Type: Regular Schedule: tue-thu@23 Minimum Blocks Shared: 1 Blocks Skipped Sharing: 0 Last Operation State: Success Last Successful Operation Begin: Thu Mar 27 23:00:00 PDT 2014 Last Successful Operation End: Fri Mar 28 07:01:44 PDT 2014 Last Operation Begin: Thu Mar 27 23:00:00 PDT 2014 Last Operation End: Fri Mar 28 07:01:44 PDT 2014 Last Operation Size: 354 GB Last Operation Error: - Change Log Usage: 2% Logical Data: 8220 GB/69 TB (11%) Queued Job: - Stale Fingerprints: 1%
na02> sis stop /vol/vm65net The operation on "/vol/vm65net" is being stopped. irt-na02> Wed Apr 2 09:35:45 PDT [irt-na02:sis.op.stopped:error]: SIS operation for /vol/vm65net has stopped
Stopped - will see if the stops the growth
thanks!
On Apr 2, 2014, at 9:32 AM, Tim Stiller tim.stiller@gmail.com wrote:
Hi Fletcher,
SIS running?
BUG 657692: Stale metadata not automatically removed during deduplication operations on volume http://support.netapp.com/NOW/cgi-bin/bol?Type=Detail&Display=657692
https://forums.netapp.com/thread/42487
regards, Tim
2014-04-02 18:24 GMT+02:00 Fletcher Cocquyt fcocquyt@stanford.edu: Hi all,
In the last 36 hours or so we have a 19Tb aggregate that is growing above 18Tb used. Usually the aggregate used level only grows if we grow its volumes. This is different - I was forced to delete snapshots and shrink volumes to get it back under 90%. And in the last 3 hours its back above 91% - used level is climbing 5-10g/minute
I so far can not see where the growth is coming from, Aggr snapshot is OFF
Ontap 8.1.2
na02> aggr status Aggr State Status Options aggr0 online raid_dp, aggr root, nosnap=on, raidsize=19 64-bit
thanks for any tips!
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters