What's changing and taking up space in snapshots?

List overview All Threads
Download

newer

older

Windows Host Utilities for Server...

NDMP dump dates

John Adams

6 Apr 2018 6 Apr '18

1:38 p.m.

Hello, Is there a way to see the changed/deleted files that are taking space in snapshots? (7 Mode, 8.2 NFS/CIFS access.)

Thanks.

Attachments:

attachment.html (text/html — 207 bytes)

Show replies by date

Ehrenwald, Ian

6 Apr 6 Apr

8:40 p.m.

As far as I am aware, you cannot do this on the filer directly.

If the data set isn't too unwieldy and you have some time to let it run, you could: 1) Create flexclone from the snapshot you want to compare against 2) Export flexclone to a Linux utility host, mount it r/o 3) Take snapshot of live filesystem 4) Create flexclone from the snapshot you just created 5) Export flexclone to the same Linux utility host, mount it r/o 5a) Recursively diff the entirety of the two flexclones (diff -urN /oldFC /newFC | tee ~/myFcDiff.txt) 5a1) This assumes everything being compared is readable text, which may not be true at all. The '-a' argument to diff could be of use, but probably not. 5b) Recursively calculate md5 hashes of files in both flexclone mountpoints (find /oldFC -xdev -type f -exec md5sum {} ; | tee ~/myOldFCHash.txt and the same for the /newFC mountpoint), then sort the files, then use either diff or comm to figure out what files have changed (or haven't)

There are probably better ways to do this but it's what popped into my head first. Hope this helps.

Ian Ehrenwald Senior Infrastructure Engineer Hachette Book Group, Inc. 1.617.263.1948 / ian.ehrenwald@hbgusa.com

________________________________________ From: toasters-bounces@teaparty.net toasters-bounces@teaparty.net on behalf of John Adams intheyc@gmail.com Sent: Friday, April 6, 2018 9:38:21 AM To: toasters@teaparty.net Subject: What's changing and taking up space in snapshots?

Hello, Is there a way to see the changed/deleted files that are taking space in snapshots? (7 Mode, 8.2 NFS/CIFS access.)

Thanks.

This may contain confidential material. If you are not an intended recipient, please notify the sender, delete immediately, and understand that no disclosure or reliance on the information herein is permitted. Hachette Book Group may monitor email to and from our network.

tmac

10:19 p.m.

Ok, a little out of the box here...I think Parisi might be proud of this one....

What about XCP?

Why not point it at a source(could be active FS or snapshot path ) and destination (same, could be the active FS or the snapshot path ) I believe it has the ability to scan and report differences. (the verify function)

Granted, not perfect, but it should be good enough to tell you what has changed and then all those files could be suspect, no?

--tmac

*Tim McCarthy, **Principal Consultant*

*Proud Member of the #NetAppATeam https://twitter.com/NetAppATeam*

*I Blog at TMACsRack https://tmacsrack.wordpress.com/*

On Fri, Apr 6, 2018 at 2:40 PM, Ehrenwald, Ian Ian.Ehrenwald@hbgusa.com wrote:

...

As far as I am aware, you cannot do this on the filer directly.

If the data set isn't too unwieldy and you have some time to let it run, you could:

Create flexclone from the snapshot you want to compare against

Export flexclone to a Linux utility host, mount it r/o

Take snapshot of live filesystem

Create flexclone from the snapshot you just created

Export flexclone to the same Linux utility host, mount it r/o

5a) Recursively diff the entirety of the two flexclones (diff -urN /oldFC /newFC | tee ~/myFcDiff.txt) 5a1) This assumes everything being compared is readable text, which may not be true at all. The '-a' argument to diff could be of use, but probably not. 5b) Recursively calculate md5 hashes of files in both flexclone mountpoints (find /oldFC -xdev -type f -exec md5sum {} ; | tee ~/myOldFCHash.txt and the same for the /newFC mountpoint), then sort the files, then use either diff or comm to figure out what files have changed (or haven't)

There are probably better ways to do this but it's what popped into my head first. Hope this helps.

Ian Ehrenwald Senior Infrastructure Engineer Hachette Book Group, Inc. 1.617.263.1948 / ian.ehrenwald@hbgusa.com

From: toasters-bounces@teaparty.net toasters-bounces@teaparty.net on behalf of John Adams intheyc@gmail.com Sent: Friday, April 6, 2018 9:38:21 AM To: toasters@teaparty.net Subject: What's changing and taking up space in snapshots?

Hello, Is there a way to see the changed/deleted files that are taking space in snapshots? (7 Mode, 8.2 NFS/CIFS access.)

Thanks.

This may contain confidential material. If you are not an intended recipient, please notify the sender, delete immediately, and understand that no disclosure or reliance on the information herein is permitted. Hachette Book Group may monitor email to and from our network.

Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters

Michael Bergman

7 Apr 7 Apr

10:04 a.m.

On 2018-04-07 00:19, tmac wrote:

...

Ok, a little out of the box here...I think Parisi might be proud of this one.... What about XCP?

I think... pride for Peter Schay, mostly ;-) No diminishing Justin, absolutely not.

XCP is good. Very good indeed. We've used it quite a bit recently around inventory and shuffling around of very large dense file trees (>20 M inodes) and it's fast, faster than anything else you can imagine. Just watch out though: with an NFS client like this one:

Xeon E5-2670 v3 @ 2.30 GHz (12-core) with 512G mem RHEL 7.2 (Maipo)

and cranking up the threads, you can pin even a big FAS box to the wall

...

Why not point it at a source(could be active FS or snapshot path ) and destination (same, could be the active FS or the snapshot path ) I believe it has the ability to scan and report differences. (the verify function)

Granted, not perfect, but it should be good enough to tell you what has changed and then all those files could be suspect, no?

--tmac

Tim McCarthy, /Principal Consultant/

Proud Member of the #NetAppATeam https://twitter.com/NetAppATeam

I Blog at TMACsRack https://tmacsrack.wordpress.com/

Parisi, Justin

8 Apr 8 Apr

4:28 p.m.

No offense taken. :)

I just tout its goodness. It’s people like Schay and Bogdan Minciu that are the real XCP MVPs. :p

Sent from my mobile device

...

On Apr 7, 2018, at 6:05 AM, Michael Bergman michael.bergman@ericsson.com wrote:

...
On 2018-04-07 00:19, tmac wrote: Ok, a little out of the box here...I think Parisi might be proud of this one.... What about XCP?

I think... pride for Peter Schay, mostly ;-) No diminishing Justin, absolutely not.

XCP is good. Very good indeed. We've used it quite a bit recently around inventory and shuffling around of very large dense file trees (>20 M inodes) and it's fast, faster than anything else you can imagine. Just watch out though: with an NFS client like this one:

Xeon E5-2670 v3 @ 2.30 GHz (12-core) with 512G mem RHEL 7.2 (Maipo)

and cranking up the threads, you can pin even a big FAS box to the wall

/M

...
Why not point it at a source(could be active FS or snapshot path ) and destination (same, could be the active FS or the snapshot path ) I believe it has the ability to scan and report differences. (the verify function)

Granted, not perfect, but it should be good enough to tell you what has changed and then all those files could be suspect, no?

--tmac

Tim McCarthy, /Principal Consultant/

Proud Member of the #NetAppATeam https://twitter.com/NetAppATeam

I Blog at TMACsRack https://tmacsrack.wordpress.com/

Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters

2521

Age (days ago)

2523

Last active (days ago)

toasters@lists.teaparty.net

4 comments

5 participants

tags (0)

participants (5)

Ehrenwald, Ian
John Adams
Michael Bergman
Parisi, Justin
tmac