Jeff,
When you say that the volumes have no space reservation, do you mean snapshot reservation, or space guarantees?
Volume sizes within an aggregate can behave very non-intuitively if they don't have any kind of space guarantee, especially if there is a mix of guaranteed and non-guaranteed volumes in that aggregate. Are there one or more "guarantee=volume" volumes in that aggregate? If any of these are growing, it will look like the available space for any non-guaranteed volumes is shrinking, which kinda looks like these non-reserved volumes are growing (which they aren't). The mass snapshot deletion may be a red herring.
Andy
----- Original Message ----- From: "Jeff Cleverley" jeff.cleverley@avagotech.com To: "Andrey Borzenkov" andrey.borzenkov@ts.fujitsu.com Cc: toasters@teaparty.net Sent: Saturday, December 17, 2011 11:48:32 AM Subject: Re: Disk space increasing after snapshots are deleted.
Andrey,
I am aware of both points. We figured it would take a while for the space to get reclaimed, but the space used in every volume in the aggregate was increasing. We know from the environment that there are daily changes made that should free up something in most of the volumes. Even if nothing did free up, we would expect no change in space utilization, not an increase in usage.
They thin provision all the volumes to 40 TB so there is no deduplication on anything. They also don't use compression of any sort that would have this type of effect. I suggested they try disabling SV for a while to let it the filer try to catch up. From what I can tell they did this and rebooted the filer. It looks like there is more space available now than there was yesterday, but it does not seem to have freed up very much space.
We still don't know why the space was increasing during the deletions.
Thanks,
Jeff
On Sat, Dec 17, 2011 at 12:56 AM, Borzenkov, Andrey andrey.borzenkov@ts.fujitsu.com wrote:
Two points
space reclamation on NetApp happens in background. For a large file it can take quite a long time.
deleting snapshot does not mean you will free anything :) Remember that snapshot shares blocks with other snapshots and active file system. So deleting snapshot simply decrements reference counts. You can estimate how much space will be actually freed *before* deleting using "snap reclaimable".
With best regards
Andrey Borzenkov Senior system engineer Service operations
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Jeff Cleverley Sent: Saturday, December 17, 2011 4:05 AM To: toasters@teaparty.net Subject: Disk space increasing after snapshots are deleted.
Greetings,
We have a 8.0.1 filer that runs as a NearStore for SV backups. The group using this normally kept 120 nightly backups. The aggregates were getting full so they decided to delete all snapshots older than 90 days. This was done with a script using snap delete <vol> <snapshot> with a list of snapshots fed into the command.
Some of the lists were not in reverse order so it deleted nightly.91 before nightly.119. I know deleting them in this type of order will work anyway, but it causes the system to do more processing.
The problem is the disk space is continuing to grow on all volumes in the aggregates. An aggr show_space of the aggregates shows everything increasing in used space. The volumes have no space reservation.
I'm trying to figure out how this mass deletion of snapshots (several hundred) is causing an increase in space usage. Any ideas?
Thanks,
Jeff