Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it. I have specifically asked what stipulation. In a VDI environment with linked clones, etc. Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.
I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or what?
Any comments welcome.
--Jordan
On Thu, Sep 22, 2016 at 02:32:30PM -0400, jordan slingerland wrote:
Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it. I have specifically asked what stipulation. In a VDI environment with linked clones, etc. Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.
I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or what?
Any comments welcome.
--Jordan
We'll be POC'ing one, so will see. Am skeptical, but don't think it's like Pure Storage where they also factor in snapshots.
Most likely the guarantee is a carrot, and if they fall short, they'll give you hardware to make up the difference. Subsequent purchases, however won't give you the same guarantee so you'll need to buy more (again, this is also what Pure does).
Other thoughts?
Ray
Well, just one (thought). Life comes w/o guarantee. :-)
Jokes aside, I agree w Ray van D; logically you can get free lunch once or twice if there's something in it for the buyer (an opportunity of *some* sort). But never will anyone buy you lunch for a whole year or more -- unless you did something that made them enter voluntary slavery
/M
On 2016-09-22 20:39, Ray Van Dolson wrote:
We'll be POC'ing one, so will see. Am skeptical, but don't think it's like Pure Storage where they also factor in snapshots.
Most likely the guarantee is a carrot, and if they fall short, they'll give you hardware to make up the difference. Subsequent purchases, however won't give you the same guarantee so you'll need to buy more (again, this is also what Pure does).
Other thoughts?
Ray _______________________________________________
On Thu, Sep 22, 2016 at 11:39:09AM -0700, Ray Van Dolson wrote:
On Thu, Sep 22, 2016 at 02:32:30PM -0400, jordan slingerland wrote:
Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it. I have specifically asked what stipulation. In a VDI environment with linked clones, etc. Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.
I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or what?
Any comments welcome.
--Jordan
We'll be POC'ing one, so will see. Am skeptical, but don't think it's like Pure Storage where they also factor in snapshots.
Most likely the guarantee is a carrot, and if they fall short, they'll give you hardware to make up the difference. Subsequent purchases, however won't give you the same guarantee so you'll need to buy more (again, this is also what Pure does).
Other thoughts?
Ray
Correction: Looking back at notes I have, the guarantee relies on the use of inline deduplication, inline compression, "compaction"[1], thin provisioning as well as factoring in "regular" volume snapshots.
NetApp will help you realize the 4:1 savings given the techniques described previously and then provide additional capacity if needed.
So, take out the snapshots and thin provisioning and will you see significantly better savings than with other technologies? Not sure -- I'd guess it'll be pretty similar.
Ray
[1] https://urldefense.proofpoint.com/v2/url?u=http-3A__www.theregister.co.uk_20...
I couldn’t find the document, but I believe NA has updated/upgraded dedupe/compression algorithms for CDOT9. I think it would be “interesting” if you could compress already compressed images/videos. They are typically not candidates for advanced storage features. I am guessing at Insight (next week in US), they will show some of this. I think the guarantee is more being in parity with other storage OEMs.
I think you would have a “discussion” with your Netapp/Sales team before you purchased anything.
http://www.netapp.com/us/forms/sales-inquiry/flash-3-4-5-promotion.aspx
On 9/22/16, 11:39 AM, "toasters-bounces@teaparty.net on behalf of Ray Van Dolson" <toasters-bounces@teaparty.net on behalf of rvandolson@esri.com> wrote:
On Thu, Sep 22, 2016 at 02:32:30PM -0400, jordan slingerland wrote: > Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. > They say they grantee it. I have specifically asked what > stipulation. In a VDI environment with linked clones, etc. Sales > guy tells me none and even specifically says they can dedup > compressed video or audio (mp3) 4:1. > > I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or > what? > > Any comments welcome. > > --Jordan
We'll be POC'ing one, so will see. Am skeptical, but don't think it's like Pure Storage where they also factor in snapshots.
Most likely the guarantee is a carrot, and if they fall short, they'll give you hardware to make up the difference. Subsequent purchases, however won't give you the same guarantee so you'll need to buy more (again, this is also what Pure does).
Other thoughts?
Ray _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
On 2016-09-22 20:54, Klise, Steve wrote:
I think it would be “interesting” if you could compress already compressed images/videos. They are typically not candidates for advanced storage features.
You cannot really, since the Shannon Entropy of such files is pretty much maxed out already.
https://en.wikipedia.org/wiki/Entropy_(information_theory)
But let's say you have a large no of smaller video files, sitting in an ONTAP (WAFL) based system. This 4K block factor, the new "Compaction" will give a positive effect.
Again: it depends
/M
my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall.
The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.
On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland < jordan.slingerland@gmail.com> wrote:
Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it. I have specifically asked what stipulation. In a VDI environment with linked clones, etc. Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.
I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or what?
Any comments welcome.
--Jordan
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
On 2016-09-22 20:44, Mike Gossett wrote:
my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall.
*duh*
The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.
Well, magic or not... the answer *must* be: it depends. Note the expressions below "if", "can be" etc.
- - - Currently Data ONTAP writes data to storage media in 4KB blocks * An I/O or file has less than 4KB of data uses an entire 4KB
Adaptive compression can compress an 8KB I/O into a 4KB block on storage * If the I/O is >50% compressible it is stored in a 4KB block – maximum storage efficiency ratio is 2:1
Storage space savings can be increased considerably if multiple small I/Os or files can be stored together in a 4KB block
* Data ONTAP 9 does this with inline data compaction * Turned on by default on AFF; can be turned on manually for hybrid systems
/M
Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else. (SSET diagnosis)
However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it.
It must transfer, as far as we're told today, via a host/file based migration. Be thinking bout this when you consider an AFF migration. It must be done outside on ONTAP.
We have pushed to get this fixed via a scanner/etc that reads and relays out the file based structure. _________________________________Jeff MohlerTech Yahoo, Storage Architect, Principal(831)454-6712 YPAC Gold Member Twitter: @PrincipalYahoo CorpIM: Hipchat & Iris
On Friday, September 23, 2016 1:53 AM, Mike Gossett cmgossett@gmail.com wrote:
my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall. The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.
On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland jordan.slingerland@gmail.com wrote:
Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it. I have specifically asked what stipulation. In a VDI environment with linked clones, etc. Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.
I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or what?
Any comments welcome.
--Jordan
______________________________ _________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/ mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
On 2016-09-23 09:09, Jeffrey Mohler wrote:
Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else. (SSET diagnosis)
I'm not too surprised (w.r.t. Yahoo). I expect much the same for us here
However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it.
It must transfer, as far as we're told today, via a host/file based migration. Be thinking bout this when you consider an AFF migration. It must be done outside on ONTAP.
We have pushed to get this [having to file migrate via protocol] fixed via a scanner/etc that reads and relays out the file based structure.
We = Yahoo? You've requested NetApp to implement a bg WAFL scanner which will runt through the whole FlexVol (vol by vol) and compact it? While this will prob be fine (-ish) for AFF I don't think that scanner will be pleasant, even viable, to run on HyA based systems (spinning disk). If they do this my hunch is it will be limited to run in AFF only...
/M
blockquote, div.yahoo_quoted { margin-left: 0 !important; border-left:1px #715FFA solid !important; padding-left:1ex !important; background-color:white !important; } Yes I did. If I want to and it reduces my costs by 20%, I want that OPTION. If you could run that across say 600PB of data, as one worlkload example for said 20%, you wouldn't? Even on dense sata only. I would want that option. Gaining even 10% for data at moderate rest for with nearly zero performance guarantees against it would be worth the week, two, or so of scanning. It's Netapp's job to make it viable and compete. Mine (ours) to guide them (and others)on what we all want.
Sent from Yahoo Mail for iPhone
On Friday, September 23, 2016, 3:45 PM, Michael Bergman michael.bergman@ericsson.com wrote:
On 2016-09-23 09:09, Jeffrey Mohler wrote:
Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else. (SSET diagnosis)
I'm not too surprised (w.r.t. Yahoo). I expect much the same for us here
However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it.
It must transfer, as far as we're told today, via a host/file based migration. Be thinking bout this when you consider an AFF migration. It must be done outside on ONTAP.
We have pushed to get this [having to file migrate via protocol] fixed via a scanner/etc that reads and relays out the file based structure.
We = Yahoo? You've requested NetApp to implement a bg WAFL scanner which will runt through the whole FlexVol (vol by vol) and compact it? While this will prob be fine (-ish) for AFF I don't think that scanner will be pleasant, even viable, to run on HyA based systems (spinning disk). If they do this my hunch is it will be limited to run in AFF only...
/M
_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
A vol-move operation will cause compaction to happen. I know it's not ideal but at least its internal. Obviously a scanner is preferred but functionally it would be a lot like a vol move.
Likewise if you use FLI to import a LUN that will trigger all the efficiency features during the import.
Sent from my mobile phone.
On 23 Sep 2016, at 02:10, Jeffrey Mohler <jmohler@yahoo-inc.commailto:jmohler@yahoo-inc.com> wrote:
Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else. (SSET diagnosis)
However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it.
It must transfer, as far as we're told today, via a host/file based migration. Be thinking bout this when you consider an AFF migration. It must be done outside on ONTAP.
We have pushed to get this fixed via a scanner/etc that reads and relays out the file based structure.
_________________________________ Jeff Mohlermailto:jmohler@yahoo-inc.com Tech Yahoo, Storage Architect, Principal (831)454-6712 YPAC Gold Member Twitter: @PrincipalYahoo CorpIM: Hipchat & Iris
On Friday, September 23, 2016 1:53 AM, Mike Gossett <cmgossett@gmail.commailto:cmgossett@gmail.com> wrote:
my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall.
The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.
On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland <jordan.slingerland@gmail.commailto:jordan.slingerland@gmail.com> wrote: Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it. I have specifically asked what stipulation. In a VDI environment with linked clones, etc. Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.
I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or what?
Any comments welcome.
--Jordan
______________________________ _________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/ mailman/listinfo/toastershttp://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Strange how compaction appears to work with vol move but not with snap mirror.
.
On Sep 23, 2016, at 4:39 AM, Steiner, Jeffrey <Jeffrey.Steiner@netapp.commailto:Jeffrey.Steiner@netapp.com> wrote:
A vol-move operation will cause compaction to happen. I know it's not ideal but at least its internal. Obviously a scanner is preferred but functionally it would be a lot like a vol move.
Likewise if you use FLI to import a LUN that will trigger all the efficiency features during the import.
Sent from my mobile phone.
On 23 Sep 2016, at 02:10, Jeffrey Mohler <jmohler@yahoo-inc.commailto:jmohler@yahoo-inc.com> wrote:
Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else. (SSET diagnosis)
However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it.
It must transfer, as far as we're told today, via a host/file based migration. Be thinking bout this when you consider an AFF migration. It must be done outside on ONTAP.
We have pushed to get this fixed via a scanner/etc that reads and relays out the file based structure.
_________________________________ Jeff Mohlermailto:jmohler@yahoo-inc.com Tech Yahoo, Storage Architect, Principal (831)454-6712 YPAC Gold Member Twitter: @PrincipalYahoo CorpIM: Hipchat & Iris
On Friday, September 23, 2016 1:53 AM, Mike Gossett <cmgossett@gmail.commailto:cmgossett@gmail.com> wrote:
my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall.
The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.
On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland <jordan.slingerland@gmail.commailto:jordan.slingerland@gmail.com> wrote: Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it. I have specifically asked what stipulation. In a VDI environment with linked clones, etc. Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.
I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or what?
Any comments welcome.
--Jordan
______________________________ _________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/ mailman/listinfo/toastershttp://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters _______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
My understanding is that both are block based operations so it makes sense to me that the blocks would be put down on disk unchanged. Perhaps it has something to do with the fact that you may want to reverse the snapmirror back to the system that presumably does not have compaction enabled or support the feature, so the blocks are left uncompacted? If the vol move is within the same controller AFF the lack of backward compatibility is not a concern. Just a though, I don't know.
On Fri, Sep 23, 2016 at 10:57 AM, Francis Kim fkim@berkcom.com wrote:
Strange how compaction appears to work with vol move but not with snap mirror.
.
On Sep 23, 2016, at 4:39 AM, Steiner, Jeffrey Jeffrey.Steiner@netapp.com wrote:
A vol-move operation will cause compaction to happen. I know it's not ideal but at least its internal. Obviously a scanner is preferred but functionally it would be a lot like a vol move.
Likewise if you use FLI to import a LUN that will trigger all the efficiency features during the import.
Sent from my mobile phone.
On 23 Sep 2016, at 02:10, Jeffrey Mohler jmohler@yahoo-inc.com wrote:
Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else. (SSET diagnosis)
However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it.
It must transfer, as far as we're told today, via a host/file based migration. Be thinking bout this when you consider an AFF migration. It must be done outside on ONTAP.
We have pushed to get this fixed via a scanner/etc that reads and relays out the file based structure.
Jeff Mohler jmohler@yahoo-inc.com Tech Yahoo, Storage Architect, Principal (831)454-6712 YPAC Gold Member Twitter: @PrincipalYahoo CorpIM: Hipchat & Iris
On Friday, September 23, 2016 1:53 AM, Mike Gossett cmgossett@gmail.com wrote:
my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall.
The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.
On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland < jordan.slingerland@gmail.com> wrote:
Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it. I have specifically asked what stipulation. In a VDI environment with linked clones, etc. Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.
I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or what?
Any comments welcome.
--Jordan
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/ mailman/listinfo/toasters http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
I would guess that snapmirror is currently looking at the physical blocks on disk, whereas a vol-move is a step up the chain and it can look at block that are actually a block within a block.
I'll check with engineering and report back...
From: jordan slingerland [mailto:jordan.slingerland@gmail.com] Sent: Friday, September 23, 2016 10:26 AM To: NGC-fkim-berkcom.com fkim@berkcom.com Cc: Steiner, Jeffrey Jeffrey.Steiner@netapp.com; toasters@teaparty.net Subject: Re: AFF and 4:1 guaranteed efficiency
My understanding is that both are block based operations so it makes sense to me that the blocks would be put down on disk unchanged. Perhaps it has something to do with the fact that you may want to reverse the snapmirror back to the system that presumably does not have compaction enabled or support the feature, so the blocks are left uncompacted? If the vol move is within the same controller AFF the lack of backward compatibility is not a concern. Just a though, I don't know.
On Fri, Sep 23, 2016 at 10:57 AM, Francis Kim <fkim@berkcom.commailto:fkim@berkcom.com> wrote: Strange how compaction appears to work with vol move but not with snap mirror.
.
On Sep 23, 2016, at 4:39 AM, Steiner, Jeffrey <Jeffrey.Steiner@netapp.commailto:Jeffrey.Steiner@netapp.com> wrote: A vol-move operation will cause compaction to happen. I know it's not ideal but at least its internal. Obviously a scanner is preferred but functionally it would be a lot like a vol move.
Likewise if you use FLI to import a LUN that will trigger all the efficiency features during the import.
Sent from my mobile phone.
On 23 Sep 2016, at 02:10, Jeffrey Mohler <jmohler@yahoo-inc.commailto:jmohler@yahoo-inc.com> wrote: Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else. (SSET diagnosis)
However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it. It must transfer, as far as we're told today, via a host/file based migration. Be thinking bout this when you consider an AFF migration. It must be done outside on ONTAP.
We have pushed to get this fixed via a scanner/etc that reads and relays out the file based structure.
_________________________________ Jeff Mohlermailto:jmohler@yahoo-inc.com Tech Yahoo, Storage Architect, Principal (831)454-6712 YPAC Gold Member Twitter: @PrincipalYahoo CorpIM: Hipchat & Iris
On Friday, September 23, 2016 1:53 AM, Mike Gossett <cmgossett@gmail.commailto:cmgossett@gmail.com> wrote:
my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall.
The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.
On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland <jordan.slingerland@gmail.commailto:jordan.slingerland@gmail.com> wrote: Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it. I have specifically asked what stipulation. In a VDI environment with linked clones, etc. Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1. I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or what? Any comments welcome. --Jordan
______________________________ _________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/ mailman/listinfo/toastershttp://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters _______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Okay, I heard back. Engineering says that snapmirror and vol-move should BOTH trigger compaction. If you didn't see savings, perhaps something about thin provisioning was not configured. The savings could have been there, they just weren't visible if the volume settings weren't correct.
From: Steiner, Jeffrey Sent: Friday, September 23, 2016 8:29 AM To: 'jordan slingerland' jordan.slingerland@gmail.com; NGC-fkim-berkcom.com fkim@berkcom.com Cc: toasters@teaparty.net Subject: RE: AFF and 4:1 guaranteed efficiency
I would guess that snapmirror is currently looking at the physical blocks on disk, whereas a vol-move is a step up the chain and it can look at block that are actually a block within a block.
I'll check with engineering and report back...
From: jordan slingerland [mailto:jordan.slingerland@gmail.com] Sent: Friday, September 23, 2016 10:26 AM To: NGC-fkim-berkcom.com <fkim@berkcom.commailto:fkim@berkcom.com> Cc: Steiner, Jeffrey <Jeffrey.Steiner@netapp.commailto:Jeffrey.Steiner@netapp.com>; toasters@teaparty.netmailto:toasters@teaparty.net Subject: Re: AFF and 4:1 guaranteed efficiency
My understanding is that both are block based operations so it makes sense to me that the blocks would be put down on disk unchanged. Perhaps it has something to do with the fact that you may want to reverse the snapmirror back to the system that presumably does not have compaction enabled or support the feature, so the blocks are left uncompacted? If the vol move is within the same controller AFF the lack of backward compatibility is not a concern. Just a though, I don't know.
On Fri, Sep 23, 2016 at 10:57 AM, Francis Kim <fkim@berkcom.commailto:fkim@berkcom.com> wrote: Strange how compaction appears to work with vol move but not with snap mirror.
.
On Sep 23, 2016, at 4:39 AM, Steiner, Jeffrey <Jeffrey.Steiner@netapp.commailto:Jeffrey.Steiner@netapp.com> wrote: A vol-move operation will cause compaction to happen. I know it's not ideal but at least its internal. Obviously a scanner is preferred but functionally it would be a lot like a vol move.
Likewise if you use FLI to import a LUN that will trigger all the efficiency features during the import.
Sent from my mobile phone.
On 23 Sep 2016, at 02:10, Jeffrey Mohler <jmohler@yahoo-inc.commailto:jmohler@yahoo-inc.com> wrote: Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else. (SSET diagnosis)
However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it. It must transfer, as far as we're told today, via a host/file based migration. Be thinking bout this when you consider an AFF migration. It must be done outside on ONTAP.
We have pushed to get this fixed via a scanner/etc that reads and relays out the file based structure.
_________________________________ Jeff Mohlermailto:jmohler@yahoo-inc.com Tech Yahoo, Storage Architect, Principal (831)454-6712 YPAC Gold Member Twitter: @PrincipalYahoo CorpIM: Hipchat & Iris
On Friday, September 23, 2016 1:53 AM, Mike Gossett <cmgossett@gmail.commailto:cmgossett@gmail.com> wrote:
my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall.
The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.
On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland <jordan.slingerland@gmail.commailto:jordan.slingerland@gmail.com> wrote: Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it. I have specifically asked what stipulation. In a VDI environment with linked clones, etc. Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1. I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or what? Any comments welcome. --Jordan
______________________________ _________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/ mailman/listinfo/toastershttp://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters _______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
For the snap mirror to trigger compaction, the aggregate must have compaction enabled. Otherwise, it is just a SnapMirror transfer
Get Outlook for iOS
On Mon, Sep 26, 2016 at 11:04 AM -0700, "Steiner, Jeffrey" Jeffrey.Steiner@netapp.com wrote:
Okay, I heard back. Engineering says that snapmirror and vol-move should BOTH trigger compaction. If you didn't see savings, perhaps something about thin provisioning was not configured. The savings could have been there, they just weren't visible if the volume settings weren't correct.
From: Steiner, Jeffrey
Sent: Friday, September 23, 2016 8:29 AM
To: 'jordan slingerland' jordan.slingerland@gmail.com; NGC-fkim-berkcom.com fkim@berkcom.com
Cc: toasters@teaparty.net
Subject: RE: AFF and 4:1 guaranteed efficiency
I would guess that snapmirror is currently looking at the physical blocks on disk, whereas a vol-move is a step up the chain and it can look at block that are actually a block within a block.
I'll check with engineering and report back...
From: jordan slingerland [mailto:jordan.slingerland@gmail.com]
Sent: Friday, September 23, 2016 10:26 AM
To: NGC-fkim-berkcom.com fkim@berkcom.com
Cc: Steiner, Jeffrey Jeffrey.Steiner@netapp.com; toasters@teaparty.net
Subject: Re: AFF and 4:1 guaranteed efficiency
My understanding is that both are block based operations so it makes sense to me that the blocks would be put down on disk unchanged. Perhaps it has something to do with the fact that you may want to reverse the snapmirror back to the system that presumably does not have compaction enabled or support the feature, so the blocks are left uncompacted? If the vol move is within the same controller AFF the lack of backward compatibility is not a concern. Just a though, I don't know.
On Fri, Sep 23, 2016 at 10:57 AM, Francis Kim fkim@berkcom.com wrote:
Strange how compaction appears to work with vol move but not with snap mirror.
.
On Sep 23, 2016, at 4:39 AM, Steiner, Jeffrey Jeffrey.Steiner@netapp.com wrote:
A vol-move operation will cause compaction to happen. I know it's not ideal but at least its internal. Obviously a scanner is preferred but functionally it would be a lot like a vol move.
Likewise if you use FLI to import a LUN that will trigger all the efficiency features during the import.
Sent from my mobile phone.
On 23 Sep 2016, at 02:10, Jeffrey Mohler jmohler@yahoo-inc.com wrote:
Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else. (SSET diagnosis)
However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it.
It must transfer, as far as we're told today, via a host/file based migration. Be thinking bout this when you consider an AFF migration. It must be done outside on ONTAP.
We have pushed to get this fixed via a scanner/etc that reads and relays out the file based structure.
_________________________________
Jeff Mohler
Tech Yahoo, Storage Architect, Principal
(831)454-6712
YPAC Gold Member
Twitter: @PrincipalYahoo
CorpIM: Hipchat & Iris
On Friday, September 23, 2016 1:53 AM, Mike Gossett cmgossett@gmail.com wrote:
my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall.
The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.
On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland jordan.slingerland@gmail.com wrote:
Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it. I have specifically asked what stipulation. In a VDI environment with linked clones, etc. Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.
I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or what?
Any comments welcome.
--Jordan
______________________________ _________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/ mailman/listinfo/toasters
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
Compaction setting is done at the volume, and not the aggregate level.
Francis Kim Cell: 415-606-2525 Direct: 510-644-1599 x334 fkim@berkcom.commailto:fkim@berkcom.com www.berkcom.comhttp://www.berkcom.com
On Sep 26, 2016, at 12:29 PM, Tim McCarthy <tmacmd@gmail.commailto:tmacmd@gmail.com> wrote:
For the snap mirror to trigger compaction, the aggregate must have compaction enabled. Otherwise, it is just a SnapMirror transfer
Get Outlook for iOShttps://aka.ms/o0ukef
On Mon, Sep 26, 2016 at 11:04 AM -0700, "Steiner, Jeffrey" <Jeffrey.Steiner@netapp.commailto:Jeffrey.Steiner@netapp.com> wrote:
Okay, I heard back. Engineering says that snapmirror and vol-move should BOTH trigger compaction. If you didn't see savings, perhaps something about thin provisioning was not configured. The savings could have been there, they just weren't visible if the volume settings weren't correct.
From: Steiner, Jeffrey Sent: Friday, September 23, 2016 8:29 AM To: 'jordan slingerland' <jordan.slingerland@gmail.commailto:jordan.slingerland@gmail.com>; NGC-fkim-berkcom.comhttp://ngc-fkim-berkcom.com/ <fkim@berkcom.commailto:fkim@berkcom.com> Cc: toasters@teaparty.netmailto:toasters@teaparty.net Subject: RE: AFF and 4:1 guaranteed efficiency
I would guess that snapmirror is currently looking at the physical blocks on disk, whereas a vol-move is a step up the chain and it can look at block that are actually a block within a block.
I'll check with engineering and report back...
From: jordan slingerland [mailto:jordan.slingerland@gmail.com] Sent: Friday, September 23, 2016 10:26 AM To: NGC-fkim-berkcom.comhttp://ngc-fkim-berkcom.com/ <fkim@berkcom.commailto:fkim@berkcom.com> Cc: Steiner, Jeffrey <Jeffrey.Steiner@netapp.commailto:Jeffrey.Steiner@netapp.com>; toasters@teaparty.netmailto:toasters@teaparty.net Subject: Re: AFF and 4:1 guaranteed efficiency
My understanding is that both are block based operations so it makes sense to me that the blocks would be put down on disk unchanged. Perhaps it has something to do with the fact that you may want to reverse the snapmirror back to the system that presumably does not have compaction enabled or support the feature, so the blocks are left uncompacted? If the vol move is within the same controller AFF the lack of backward compatibility is not a concern. Just a though, I don't know.
On Fri, Sep 23, 2016 at 10:57 AM, Francis Kim <fkim@berkcom.commailto:fkim@berkcom.com> wrote: Strange how compaction appears to work with vol move but not with snap mirror.
.
On Sep 23, 2016, at 4:39 AM, Steiner, Jeffrey <Jeffrey.Steiner@netapp.commailto:Jeffrey.Steiner@netapp.com> wrote: A vol-move operation will cause compaction to happen. I know it's not ideal but at least its internal. Obviously a scanner is preferred but functionally it would be a lot like a vol move.
Likewise if you use FLI to import a LUN that will trigger all the efficiency features during the import.
Sent from my mobile phone.
On 23 Sep 2016, at 02:10, Jeffrey Mohler <jmohler@yahoo-inc.commailto:jmohler@yahoo-inc.com> wrote: Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else. (SSET diagnosis)
However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it. It must transfer, as far as we're told today, via a host/file based migration. Be thinking bout this when you consider an AFF migration. It must be done outside on ONTAP.
We have pushed to get this fixed via a scanner/etc that reads and relays out the file based structure.
_________________________________ Jeff Mohlermailto:jmohler@yahoo-inc.com Tech Yahoo, Storage Architect, Principal (831)454-6712 YPAC Gold Member Twitter: @PrincipalYahoo CorpIM: Hipchat & Iris
On Friday, September 23, 2016 1:53 AM, Mike Gossett <cmgossett@gmail.commailto:cmgossett@gmail.com> wrote:
my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall.
The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.
On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland <jordan.slingerland@gmail.commailto:jordan.slingerland@gmail.com> wrote: Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it. I have specifically asked what stipulation. In a VDI environment with linked clones, etc. Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1. I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or what? Any comments welcome. --Jordan
______________________________ _________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/ mailman/listinfo/toastershttp://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters _______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Have to disagree. I pulled up an ONTAP 9 simulator and did this exact test. My source was a non compacted volume. Destination was non compacted. Result obviously is not compacted There is an aggregate setting which makes all volumes created on that aggregate compacted. Set that on. I reran the mirror to that aggregate and the result was compared with a non compacted source. For kicks I tried a version flexible mirror. It will not work to the aggregate with the compaction flag enabled. Makes sense since only DP mirrors support compaction
Get Outlook for iOS
On Mon, Sep 26, 2016 at 1:15 PM -0700, "Francis Kim" fkim@berkcom.com wrote:
Compaction setting is done at the volume, and not the aggregate level.
Francis Kim
Cell: 415-606-2525
Direct: 510-644-1599 x334
fkim@berkcom.com
www.berkcom.com
On Sep 26, 2016, at 12:29 PM, Tim McCarthy tmacmd@gmail.com wrote:
For the snap mirror to trigger compaction, the aggregate must have compaction enabled. Otherwise, it is just a SnapMirror transfer
Get Outlook for iOS
On Mon, Sep 26, 2016 at 11:04 AM -0700, "Steiner, Jeffrey" Jeffrey.Steiner@netapp.com wrote:
Okay, I heard back. Engineering says that snapmirror and vol-move should BOTH trigger compaction. If you didn't see savings, perhaps something about thin provisioning was not configured. The savings could have been there, they just weren't visible if the volume settings weren't correct.
From: Steiner, Jeffrey
Sent: Friday, September 23, 2016 8:29 AM
To: 'jordan slingerland' jordan.slingerland@gmail.com; NGC-fkim-berkcom.com fkim@berkcom.com
Cc: toasters@teaparty.net
Subject: RE: AFF and 4:1 guaranteed efficiency
I would guess that snapmirror is currently looking at the physical blocks on disk, whereas a vol-move is a step up the chain and it can look at block that are actually a block within a block.
I'll check with engineering and report back...
From: jordan slingerland [mailto:jordan.slingerland@gmail.com]
Sent: Friday, September 23, 2016 10:26 AM
To: NGC-fkim-berkcom.com fkim@berkcom.com
Cc: Steiner, Jeffrey Jeffrey.Steiner@netapp.com; toasters@teaparty.net
Subject: Re: AFF and 4:1 guaranteed efficiency
My understanding is that both are block based operations so it makes sense to me that the blocks would be put down on disk unchanged. Perhaps it has something to do with the fact that you may want to reverse the snapmirror back to the system that presumably does not have compaction enabled or support the feature, so the blocks are left uncompacted? If the vol move is within the same controller AFF the lack of backward compatibility is not a concern. Just a though, I don't know.
On Fri, Sep 23, 2016 at 10:57 AM, Francis Kim fkim@berkcom.com wrote:
Strange how compaction appears to work with vol move but not with snap mirror.
.
On Sep 23, 2016, at 4:39 AM, Steiner, Jeffrey Jeffrey.Steiner@netapp.com wrote:
A vol-move operation will cause compaction to happen. I know it's not ideal but at least its internal. Obviously a scanner is preferred but functionally it would be a lot like a vol move.
Likewise if you use FLI to import a LUN that will trigger all the efficiency features during the import.
Sent from my mobile phone.
On 23 Sep 2016, at 02:10, Jeffrey Mohler jmohler@yahoo-inc.com wrote:
Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else. (SSET diagnosis)
However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it.
It must transfer, as far as we're told today, via a host/file based migration. Be thinking bout this when you consider an AFF migration. It must be done outside on ONTAP.
We have pushed to get this fixed via a scanner/etc that reads and relays out the file based structure.
_________________________________
Jeff Mohler
Tech Yahoo, Storage Architect, Principal
(831)454-6712
YPAC Gold Member
Twitter: @PrincipalYahoo
CorpIM: Hipchat & Iris
On Friday, September 23, 2016 1:53 AM, Mike Gossett cmgossett@gmail.com wrote:
my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall.
The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.
On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland jordan.slingerland@gmail.com wrote:
Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it. I have specifically asked what stipulation. In a VDI environment with linked clones, etc. Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.
I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or what?
Any comments welcome.
--Jordan
______________________________ _________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/ mailman/listinfo/toasters
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
There are options on aggregate and volume levels. Compaction is enabled by default on aggregate and volume level for AFF. But in FAS you should enable it for aggregate. And after that all new volumes with space guarantee none will have compaction enabled. Here is more information from "Logical Storage Management Guide":
You can control inline data compaction on FAS systems with Flash Pool (hybrid) or HDD aggregates at the volume or aggregate level using the wafl compaction enable node shell command. Data compaction is disabled by default for FAS systems.
About this task
If you enable data compaction at the aggregate level, any new volume created with a volume space guarantee of none in the aggregate has data compaction enabled. Existing volumes are not enabled. Enabling data compaction on a volume on an HDD aggregate uses additional CPU resources.
Steps
1.
Change to the advanced privilege level: set -privilege advanced 2.
Check the data compaction state of volumes and aggregates for the desired node: node run –node node_name –command wafl compaction status -all 3.
Enable data compaction on the aggregate or volume. -
For aggregates:
node run –node node_name –command wafl compaction enable –A aggregate_name
-
For volumes:
node run –node node_name –command wafl compaction enable volume_name 4.
View the compaction statistics:
node run –node node_name –command wafl compaction stats
On Mon, Sep 26, 2016 at 11:24 PM, Tim McCarthy tmacmd@gmail.com wrote:
Have to disagree. I pulled up an ONTAP 9 simulator and did this exact test.
My source was a non compacted volume. Destination was non compacted. Result obviously is not compacted
There is an aggregate setting which makes all volumes created on that aggregate compacted. Set that on. I reran the mirror to that aggregate and the result was compared with a non compacted source.
For kicks I tried a version flexible mirror. It will not work to the aggregate with the compaction flag enabled.
Makes sense since only DP mirrors support compaction
Get Outlook for iOS https://aka.ms/o0ukef
On Mon, Sep 26, 2016 at 1:15 PM -0700, "Francis Kim" fkim@berkcom.com wrote:
Compaction setting is done at the volume, and not the aggregate level.
Francis Kim Cell: 415-606-2525 Direct: 510-644-1599 x334 fkim@berkcom.com www.berkcom.com
On Sep 26, 2016, at 12:29 PM, Tim McCarthy tmacmd@gmail.com wrote:
For the snap mirror to trigger compaction, the aggregate must have compaction enabled. Otherwise, it is just a SnapMirror transfer
Get Outlook for iOS https://aka.ms/o0ukef
On Mon, Sep 26, 2016 at 11:04 AM -0700, "Steiner, Jeffrey" < Jeffrey.Steiner@netapp.com> wrote:
Okay, I heard back. Engineering says that snapmirror and vol-move should
BOTH trigger compaction. If you didn't see savings, perhaps something about thin provisioning was not configured. The savings could have been there, they just weren't visible if the volume settings weren't correct.
*From:* Steiner, Jeffrey *Sent:* Friday, September 23, 2016 8:29 AM *To:* 'jordan slingerland' jordan.slingerland@gmail.com; NGC-fkim-berkcom.com http://ngc-fkim-berkcom.com/ fkim@berkcom.com *Cc:* toasters@teaparty.net *Subject:* RE: AFF and 4:1 guaranteed efficiency
I would guess that snapmirror is currently looking at the physical blocks on disk, whereas a vol-move is a step up the chain and it can look at block that are actually a block within a block.
I'll check with engineering and report back...
*From:* jordan slingerland [mailto:jordan.slingerland@gmail.com jordan.slingerland@gmail.com] *Sent:* Friday, September 23, 2016 10:26 AM *To:* NGC-fkim-berkcom.com http://ngc-fkim-berkcom.com/ <fkim @berkcom.com> *Cc:* Steiner, Jeffrey Jeffrey.Steiner@netapp.com; toasters@teaparty.net *Subject:* Re: AFF and 4:1 guaranteed efficiency
My understanding is that both are block based operations so it makes sense to me that the blocks would be put down on disk unchanged. Perhaps it has something to do with the fact that you may want to reverse the snapmirror back to the system that presumably does not have compaction enabled or support the feature, so the blocks are left uncompacted? If the vol move is within the same controller AFF the lack of backward compatibility is not a concern. Just a though, I don't know.
On Fri, Sep 23, 2016 at 10:57 AM, Francis Kim fkim@berkcom.com wrote:
Strange how compaction appears to work with vol move but not with snap mirror.
.
On Sep 23, 2016, at 4:39 AM, Steiner, Jeffrey < Jeffrey.Steiner@netapp.com> wrote:
A vol-move operation will cause compaction to happen. I know it's not ideal but at least its internal. Obviously a scanner is preferred but functionally it would be a lot like a vol move.
Likewise if you use FLI to import a LUN that will trigger all the efficiency features during the import.
Sent from my mobile phone.
On 23 Sep 2016, at 02:10, Jeffrey Mohler jmohler@yahoo-inc.com wrote:
Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else. (SSET diagnosis)
However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it. It must transfer, as far as we're told today, via a host/file based migration. Be thinking bout this when you consider an AFF migration. It must be done outside on ONTAP.
We have pushed to get this fixed via a scanner/etc that reads and relays out the file based structure.
Jeff Mohler jmohler@yahoo-inc.com Tech Yahoo, Storage Architect, Principal (831)454-6712 YPAC Gold Member Twitter: @PrincipalYahoo CorpIM: Hipchat & Iris
On Friday, September 23, 2016 1:53 AM, Mike Gossett cmgossett@gmail.com wrote:
my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall.
The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.
On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland < jordan.slingerland@gmail.com> wrote:
Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it. I have specifically asked what stipulation. In a VDI environment with linked clones, etc. Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.
I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or what?
Any comments welcome. --Jordan
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/ mailman/listinfo/toasters http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Right. And ONTAP will not allow you to drop a destination mirror based on XDP or LS into an aggregate with compaction enabled. Not 100% sure about TDP as I did not try that. Maybe it will work. will have to create a couple of SIMs and give it the old college try.
--tmac
*Tim McCarthy, **Principal Consultant*
*Proud Member of the #NetAppATeam https://twitter.com/NetAppATeam*
*I Blog at TMACsRack https://tmacsrack.wordpress.com/*
443-228-TMAC (*Google Voice*) 214-279-3926 (*eFAX*)
[image: NetApp - In partnership with Alpine Testing Solutions][image: NetApp Certified Data Administrator, ONTAP] https://www.certmetrics.com/netapp/public/badge.aspx?t=c&d=2012-11-05&i=35&ci=NETAPP00041276[image: NetApp Certified Implementation Engineer - SAN Specialist, ONTAP] https://www.certmetrics.com/netapp/public/badge.aspx?t=c&d=2012-11-08&i=36&ci=NETAPP00041276[image: NetApp Certified Storage Installation Engineer, ONTAP] https://www.certmetrics.com/netapp/public/badge.aspx?t=c&d=2015-10-13&i=38&ci=NETAPP00041276[image: NetApp Certified Implementation Engineer - Data Protection Specialist] https://www.certmetrics.com/netapp/public/badge.aspx?t=c&d=2015-10-15&i=11&ci=NETAPP00041276
NetApp Candidate ID: NETAPP00041276 FlexPod Design: Oct 2015 - Jan 2018, S0N62WE1BMVEYF3M FlexPod Implementation: Oct 2015 - Jan 2018, JH3QJT4KLEQ41HPH
RHCE6 110-107-141 https://www.redhat.com/wapps/training/certification/verify.html?certNumber=110-107-141&isSearch=False&verify=Verify
On Mon, Sep 26, 2016 at 4:52 PM, Arthur Alikulov a.alikulov@gmail.com wrote:
There are options on aggregate and volume levels. Compaction is enabled by default on aggregate and volume level for AFF. But in FAS you should enable it for aggregate. And after that all new volumes with space guarantee none will have compaction enabled. Here is more information from "Logical Storage Management Guide":
You can control inline data compaction on FAS systems with Flash Pool (hybrid) or HDD aggregates at the volume or aggregate level using the wafl compaction enable node shell command. Data compaction is disabled by default for FAS systems.
About this task
If you enable data compaction at the aggregate level, any new volume created with a volume space guarantee of none in the aggregate has data compaction enabled. Existing volumes are not enabled. Enabling data compaction on a volume on an HDD aggregate uses additional CPU resources.
Steps
Change to the advanced privilege level: set -privilege advanced 2.
Check the data compaction state of volumes and aggregates for the desired node: node run –node node_name –command wafl compaction status -all 3.
Enable data compaction on the aggregate or volume.
For aggregates:
node run –node node_name –command wafl compaction enable –A aggregate_name
For volumes:
node run –node node_name –command wafl compaction enable volume_name
View the compaction statistics:
node run –node node_name –command wafl compaction stats
On Mon, Sep 26, 2016 at 11:24 PM, Tim McCarthy tmacmd@gmail.com wrote:
Have to disagree. I pulled up an ONTAP 9 simulator and did this exact test.
My source was a non compacted volume. Destination was non compacted. Result obviously is not compacted
There is an aggregate setting which makes all volumes created on that aggregate compacted. Set that on. I reran the mirror to that aggregate and the result was compared with a non compacted source.
For kicks I tried a version flexible mirror. It will not work to the aggregate with the compaction flag enabled.
Makes sense since only DP mirrors support compaction
Get Outlook for iOS https://aka.ms/o0ukef
On Mon, Sep 26, 2016 at 1:15 PM -0700, "Francis Kim" fkim@berkcom.com wrote:
Compaction setting is done at the volume, and not the aggregate level.
Francis Kim Cell: 415-606-2525 Direct: 510-644-1599 x334 fkim@berkcom.com www.berkcom.com
On Sep 26, 2016, at 12:29 PM, Tim McCarthy tmacmd@gmail.com wrote:
For the snap mirror to trigger compaction, the aggregate must have compaction enabled. Otherwise, it is just a SnapMirror transfer
Get Outlook for iOS https://aka.ms/o0ukef
On Mon, Sep 26, 2016 at 11:04 AM -0700, "Steiner, Jeffrey" < Jeffrey.Steiner@netapp.com> wrote:
Okay, I heard back. Engineering says that snapmirror and vol-move should
BOTH trigger compaction. If you didn't see savings, perhaps something about thin provisioning was not configured. The savings could have been there, they just weren't visible if the volume settings weren't correct.
*From:* Steiner, Jeffrey *Sent:* Friday, September 23, 2016 8:29 AM *To:* 'jordan slingerland' jordan.slingerland@gmail.com; NGC-fkim-berkcom.com http://ngc-fkim-berkcom.com/ fkim@berkcom.com *Cc:* toasters@teaparty.net *Subject:* RE: AFF and 4:1 guaranteed efficiency
I would guess that snapmirror is currently looking at the physical blocks on disk, whereas a vol-move is a step up the chain and it can look at block that are actually a block within a block.
I'll check with engineering and report back...
*From:* jordan slingerland [mailto:jordan.slingerland@gmail.com jordan.slingerland@gmail.com] *Sent:* Friday, September 23, 2016 10:26 AM *To:* NGC-fkim-berkcom.com http://ngc-fkim-berkcom.com/ <fkim @berkcom.com> *Cc:* Steiner, Jeffrey Jeffrey.Steiner@netapp.com; toasters@teaparty.net *Subject:* Re: AFF and 4:1 guaranteed efficiency
My understanding is that both are block based operations so it makes sense to me that the blocks would be put down on disk unchanged. Perhaps it has something to do with the fact that you may want to reverse the snapmirror back to the system that presumably does not have compaction enabled or support the feature, so the blocks are left uncompacted? If the vol move is within the same controller AFF the lack of backward compatibility is not a concern. Just a though, I don't know.
On Fri, Sep 23, 2016 at 10:57 AM, Francis Kim fkim@berkcom.com wrote:
Strange how compaction appears to work with vol move but not with snap mirror.
.
On Sep 23, 2016, at 4:39 AM, Steiner, Jeffrey < Jeffrey.Steiner@netapp.com> wrote:
A vol-move operation will cause compaction to happen. I know it's not ideal but at least its internal. Obviously a scanner is preferred but functionally it would be a lot like a vol move.
Likewise if you use FLI to import a LUN that will trigger all the efficiency features during the import.
Sent from my mobile phone.
On 23 Sep 2016, at 02:10, Jeffrey Mohler jmohler@yahoo-inc.com wrote:
Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else. (SSET diagnosis)
However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it. It must transfer, as far as we're told today, via a host/file based migration. Be thinking bout this when you consider an AFF migration. It must be done outside on ONTAP.
We have pushed to get this fixed via a scanner/etc that reads and relays out the file based structure.
Jeff Mohler jmohler@yahoo-inc.com Tech Yahoo, Storage Architect, Principal (831)454-6712 YPAC Gold Member Twitter: @PrincipalYahoo CorpIM: Hipchat & Iris
On Friday, September 23, 2016 1:53 AM, Mike Gossett < cmgossett@gmail.com> wrote:
my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall.
The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.
On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland < jordan.slingerland@gmail.com> wrote:
Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it. I have specifically asked what stipulation. In a VDI environment with linked clones, etc. Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.
I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or what?
Any comments welcome. --Jordan
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/ mailman/listinfo/toasters http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
One more thing When you have a mirrored destination, you can't set compaction on that volume which makes sense since you can't modify the destination mirror without breaking it
Get Outlook for iOS
On Mon, Sep 26, 2016 at 1:15 PM -0700, "Francis Kim" fkim@berkcom.com wrote:
Compaction setting is done at the volume, and not the aggregate level.
Francis Kim
Cell: 415-606-2525
Direct: 510-644-1599 x334
fkim@berkcom.com
www.berkcom.com
On Sep 26, 2016, at 12:29 PM, Tim McCarthy tmacmd@gmail.com wrote:
For the snap mirror to trigger compaction, the aggregate must have compaction enabled. Otherwise, it is just a SnapMirror transfer
Get Outlook for iOS
On Mon, Sep 26, 2016 at 11:04 AM -0700, "Steiner, Jeffrey" Jeffrey.Steiner@netapp.com wrote:
Okay, I heard back. Engineering says that snapmirror and vol-move should BOTH trigger compaction. If you didn't see savings, perhaps something about thin provisioning was not configured. The savings could have been there, they just weren't visible if the volume settings weren't correct.
From: Steiner, Jeffrey
Sent: Friday, September 23, 2016 8:29 AM
To: 'jordan slingerland' jordan.slingerland@gmail.com; NGC-fkim-berkcom.com fkim@berkcom.com
Cc: toasters@teaparty.net
Subject: RE: AFF and 4:1 guaranteed efficiency
I would guess that snapmirror is currently looking at the physical blocks on disk, whereas a vol-move is a step up the chain and it can look at block that are actually a block within a block.
I'll check with engineering and report back...
From: jordan slingerland [mailto:jordan.slingerland@gmail.com]
Sent: Friday, September 23, 2016 10:26 AM
To: NGC-fkim-berkcom.com fkim@berkcom.com
Cc: Steiner, Jeffrey Jeffrey.Steiner@netapp.com; toasters@teaparty.net
Subject: Re: AFF and 4:1 guaranteed efficiency
My understanding is that both are block based operations so it makes sense to me that the blocks would be put down on disk unchanged. Perhaps it has something to do with the fact that you may want to reverse the snapmirror back to the system that presumably does not have compaction enabled or support the feature, so the blocks are left uncompacted? If the vol move is within the same controller AFF the lack of backward compatibility is not a concern. Just a though, I don't know.
On Fri, Sep 23, 2016 at 10:57 AM, Francis Kim fkim@berkcom.com wrote:
Strange how compaction appears to work with vol move but not with snap mirror.
.
On Sep 23, 2016, at 4:39 AM, Steiner, Jeffrey Jeffrey.Steiner@netapp.com wrote:
A vol-move operation will cause compaction to happen. I know it's not ideal but at least its internal. Obviously a scanner is preferred but functionally it would be a lot like a vol move.
Likewise if you use FLI to import a LUN that will trigger all the efficiency features during the import.
Sent from my mobile phone.
On 23 Sep 2016, at 02:10, Jeffrey Mohler jmohler@yahoo-inc.com wrote:
Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else. (SSET diagnosis)
However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it.
It must transfer, as far as we're told today, via a host/file based migration. Be thinking bout this when you consider an AFF migration. It must be done outside on ONTAP.
We have pushed to get this fixed via a scanner/etc that reads and relays out the file based structure.
_________________________________
Jeff Mohler
Tech Yahoo, Storage Architect, Principal
(831)454-6712
YPAC Gold Member
Twitter: @PrincipalYahoo
CorpIM: Hipchat & Iris
On Friday, September 23, 2016 1:53 AM, Mike Gossett cmgossett@gmail.com wrote:
my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall.
The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.
On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland jordan.slingerland@gmail.com wrote:
Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it. I have specifically asked what stipulation. In a VDI environment with linked clones, etc. Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.
I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or what?
Any comments welcome.
--Jordan
______________________________ _________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/ mailman/listinfo/toasters
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
blockquote, div.yahoo_quoted { margin-left: 0 !important; border-left:1px #715FFA solid !important; padding-left:1ex !important; background-color:white !important; } Great news. Beats our intro list of what works and.not. Just back from Thailand JS, landed Vegas. We'll catch up on this.
Sent from Yahoo Mail for iPhone
On Monday, September 26, 2016, 10:57 AM, Steiner, Jeffrey Jeffrey.Steiner@netapp.com wrote:
-- _filtered {panose-1:2 4 5 3 5 4 6 3 2 4;} _filtered {font-family:Calibri;panose-1:2 15 5 2 2 2 4 3 2 4;} _filtered {font-family:Verdana;panose-1:2 11 6 4 3 5 4 4 2 4;} _filtered {} p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0in;margin-bottom:.0001pt;font-size:12.0pt;}a:link, span.MsoHyperlink {color:blue;text-decoration:underline;}a:visited, span.MsoHyperlinkFollowed {color:purple;text-decoration:underline;}p.msonormal0, li.msonormal0, div.msonormal0 {margin-right:0in;margin-left:0in;font-size:12.0pt;}span.EmailStyle18 {color:windowtext;}span.EmailStyle19 {color:windowtext;}.MsoChpDefault {font-size:10.0pt;} _filtered {margin:1.0in 1.0in 1.0in 1.0in;}div.WordSection1 {} Okay, I heard back. Engineering says that snapmirror and vol-move should BOTH trigger compaction. If you didn't see savings, perhaps something about thin provisioning was not configured. The savings could have been there, they just weren't visible if the volume settings weren't correct.
From: Steiner, Jeffrey Sent: Friday, September 23, 2016 8:29 AM To: 'jordan slingerland' jordan.slingerland@gmail.com; NGC-fkim-berkcom.com fkim@berkcom.com Cc: toasters@teaparty.net Subject: RE: AFF and 4:1 guaranteed efficiency
I would guess that snapmirror is currently looking at the physical blocks on disk, whereas a vol-move is a step up the chain and it can look at block that are actually a block within a block.
I'll check with engineering and report back...
From: jordan slingerland [mailto:jordan.slingerland@gmail.com] Sent: Friday, September 23, 2016 10:26 AM To: NGC-fkim-berkcom.com fkim@berkcom.com Cc: Steiner, Jeffrey Jeffrey.Steiner@netapp.com;toasters@teaparty.net Subject: Re: AFF and 4:1 guaranteed efficiency
My understanding is that both are block based operations so it makes sense to me that the blocks would be put down on disk unchanged. Perhaps it has something to do with the fact that you may want to reverse the snapmirror back to the system that presumably does not have compaction enabled or support the feature, so the blocks are left uncompacted? If the vol move is within the same controller AFF the lack of backward compatibility is not a concern. Just a though, I don't know.
On Fri, Sep 23, 2016 at 10:57 AM, Francis Kim fkim@berkcom.com wrote:
Strange how compaction appears to work with vol move but not with snap mirror.
.
On Sep 23, 2016, at 4:39 AM, Steiner, Jeffrey Jeffrey.Steiner@netapp.com wrote:
A vol-move operation will cause compaction to happen. I know it's not ideal but at least its internal. Obviously a scanner is preferred but functionally it would be a lot like a vol move.
Likewise if you use FLI to import a LUN that will trigger all the efficiency features during the import.
Sent from my mobile phone.
On 23 Sep 2016, at 02:10, Jeffrey Mohler jmohler@yahoo-inc.com wrote:
Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else. (SSET diagnosis)
However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it.
It must transfer, as far as we're told today, via a host/file based migration. Be thinking bout this when you consider an AFF migration. It must be done outside on ONTAP.
We have pushed to get this fixed via a scanner/etc that reads and relays out the file based structure.
_________________________________
Jeff Mohler
Tech Yahoo, Storage Architect, Principal
(831)454-6712 YPAC Gold Member
Twitter: @PrincipalYahoo CorpIM: Hipchat & Iris
On Friday, September 23, 2016 1:53 AM, Mike Gossett cmgossett@gmail.com wrote:
my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall.
The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.
On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland jordan.slingerland@gmail.com wrote:
Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it. I have specifically asked what stipulation. In a VDI environment with linked clones, etc. Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.
I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or what?
Any comments welcome.
--Jordan
______________________________ _________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/ mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
I vaguely remember running a test on an ONTAP 9 simulator.
What I found was that if the destination aggr had compaction enabled, then snapmirror into that aggr would also have compacted data. This DOES NOT work with XDP (snapvault & version-flexible snapmirror)
--tmac
*Tim McCarthy, **Principal Consultant*
*Proud Member of the #NetAppATeam https://twitter.com/NetAppATeam*
*I Blog at TMACsRack https://tmacsrack.wordpress.com/*
[image: NetApp - In partnership with Alpine Testing Solutions][image: NetApp Certified Data Administrator, ONTAP] https://www.certmetrics.com/netapp/public/badge.aspx?t=c&d=2012-11-05&i=35&ci=NETAPP00041276[image: NetApp Certified Implementation Engineer - SAN Specialist, ONTAP] https://www.certmetrics.com/netapp/public/badge.aspx?t=c&d=2012-11-08&i=36&ci=NETAPP00041276[image: NetApp Certified Storage Installation Engineer, ONTAP] https://www.certmetrics.com/netapp/public/badge.aspx?t=c&d=2015-10-13&i=38&ci=NETAPP00041276[image: NetApp Certified Implementation Engineer - Data Protection Specialist] https://www.certmetrics.com/netapp/public/badge.aspx?t=c&d=2015-10-15&i=11&ci=NETAPP00041276
NetApp Candidate ID: NETAPP00041276 FlexPod Design: Oct 2015 - Jan 2018, S0N62WE1BMVEYF3M FlexPod Implementation: Oct 2015 - Jan 2018, JH3QJT4KLEQ41HPH
RHCE6 110-107-141 https://www.redhat.com/wapps/training/certification/verify.html?certNumber=110-107-141&isSearch=False&verify=Verify
On Fri, Sep 23, 2016 at 11:25 AM, jordan slingerland < jordan.slingerland@gmail.com> wrote:
My understanding is that both are block based operations so it makes sense to me that the blocks would be put down on disk unchanged. Perhaps it has something to do with the fact that you may want to reverse the snapmirror back to the system that presumably does not have compaction enabled or support the feature, so the blocks are left uncompacted? If the vol move is within the same controller AFF the lack of backward compatibility is not a concern. Just a though, I don't know.
On Fri, Sep 23, 2016 at 10:57 AM, Francis Kim fkim@berkcom.com wrote:
Strange how compaction appears to work with vol move but not with snap mirror.
.
On Sep 23, 2016, at 4:39 AM, Steiner, Jeffrey Jeffrey.Steiner@netapp.com wrote:
A vol-move operation will cause compaction to happen. I know it's not ideal but at least its internal. Obviously a scanner is preferred but functionally it would be a lot like a vol move.
Likewise if you use FLI to import a LUN that will trigger all the efficiency features during the import.
Sent from my mobile phone.
On 23 Sep 2016, at 02:10, Jeffrey Mohler jmohler@yahoo-inc.com wrote:
Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else. (SSET diagnosis)
However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it.
It must transfer, as far as we're told today, via a host/file based migration. Be thinking bout this when you consider an AFF migration. It must be done outside on ONTAP.
We have pushed to get this fixed via a scanner/etc that reads and relays out the file based structure.
Jeff Mohler jmohler@yahoo-inc.com Tech Yahoo, Storage Architect, Principal (831)454-6712 YPAC Gold Member Twitter: @PrincipalYahoo CorpIM: Hipchat & Iris
On Friday, September 23, 2016 1:53 AM, Mike Gossett cmgossett@gmail.com wrote:
my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall.
The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.
On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland < jordan.slingerland@gmail.com> wrote:
Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it. I have specifically asked what stipulation. In a VDI environment with linked clones, etc. Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.
I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or what?
Any comments welcome.
--Jordan
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/ mailman/listinfo/toasters http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
I just want to point out a few things based on my experience and feedback from other people.
Pure Storage - block only. You do not get any bells and whistles.
XIO – everyone that owns one has something bad to say about it
TinTri – NFS only, each node is a separate unit, VM aware.. has a place for certain markets
SolidFire – amazing potential, you can have disk failures or a complete node fail and stay up and running , but again, block only and no bells and whistles.
Netapp AFF – all protocols supported, the entire snap manager suite (exchange, SQL and sharepoint are absolutely outstanding) and with their 4:1 guarantee they will give you more shelves if they don’t hit that mark. They also give you controller upgrades after 3 years.
In my experience the “bake offs” have been between XIO, Pure Storage and Netapp on critical EMR systems like EPIC and McKesson. Netapp’s AFF has always been the one that came out on top as far as performance. EMC has been the one willing to give their product away just to keep their foot print.
With all that being said, you need to figure out what fits your needs. I have customers that want to boot UCS blades from SAN, Run NFS datastores and migrate their windows file servers to native CIF’s shares on their storage system. The only storage company that can do all of that is Netapp (I won’t bring up Isolation because their block size is terrible with small files)
So, if you are only worried about the 4:1 efficiencies who cares? They will give you more disks and you still get the Cadillac.
From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of tmac Sent: Friday, September 23, 2016 10:34 AM To: jordan slingerland Cc: toasters@teaparty.net Subject: Re: AFF and 4:1 guaranteed efficiency
I vaguely remember running a test on an ONTAP 9 simulator.
What I found was that if the destination aggr had compaction enabled, then snapmirror into that aggr would also have compacted data.
This DOES NOT work with XDP (snapvault & version-flexible snapmirror)
--tmac
Tim McCarthy, Principal Consultant
Proud Member of the #NetAppATeam https://twitter.com/NetAppATeam
I Blog at https://tmacsrack.wordpress.com/ TMACsRack
Image removed by sender. NetApp - In partnership with Alpine Testing Solutions https://www.certmetrics.com/netapp/public/badge.aspx?t=c&d=2012-11-05&i=35&ci=NETAPP00041276 Image removed by sender. NetApp Certified Data Administrator, ONTAP https://www.certmetrics.com/netapp/public/badge.aspx?t=c&d=2012-11-08&i=36&ci=NETAPP00041276 Image removed by sender. NetApp Certified Implementation Engineer - SAN Specialist, ONTAP https://www.certmetrics.com/netapp/public/badge.aspx?t=c&d=2015-10-13&i=38&ci=NETAPP00041276 Image removed by sender. NetApp Certified Storage Installation Engineer, ONTAP https://www.certmetrics.com/netapp/public/badge.aspx?t=c&d=2015-10-15&i=11&ci=NETAPP00041276 Image removed by sender. NetApp Certified Implementation Engineer - Data Protection SpecialistImage removed by sender.Image removed by sender.
NetApp Candidate ID: NETAPP00041276
FlexPod Design: Oct 2015 - Jan 2018, S0N62WE1BMVEYF3M
FlexPod Implementation: Oct 2015 - Jan 2018, JH3QJT4KLEQ41HPH
RHCE6 https://www.redhat.com/wapps/training/certification/verify.html?certNumber=110-107-141&isSearch=False&verify=Verify 110-107-141
On Fri, Sep 23, 2016 at 11:25 AM, jordan slingerland jordan.slingerland@gmail.com wrote:
My understanding is that both are block based operations so it makes sense to me that the blocks would be put down on disk unchanged. Perhaps it has something to do with the fact that you may want to reverse the snapmirror back to the system that presumably does not have compaction enabled or support the feature, so the blocks are left uncompacted? If the vol move is within the same controller AFF the lack of backward compatibility is not a concern. Just a though, I don't know.
On Fri, Sep 23, 2016 at 10:57 AM, Francis Kim fkim@berkcom.com wrote:
Strange how compaction appears to work with vol move but not with snap mirror.
.
On Sep 23, 2016, at 4:39 AM, Steiner, Jeffrey Jeffrey.Steiner@netapp.com wrote:
A vol-move operation will cause compaction to happen. I know it's not ideal but at least its internal. Obviously a scanner is preferred but functionally it would be a lot like a vol move.
Likewise if you use FLI to import a LUN that will trigger all the efficiency features during the import.
Sent from my mobile phone.
On 23 Sep 2016, at 02:10, Jeffrey Mohler jmohler@yahoo-inc.com wrote:
Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else. (SSET diagnosis)
However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it.
It must transfer, as far as we're told today, via a host/file based migration. Be thinking bout this when you consider an AFF migration. It must be done outside on ONTAP.
We have pushed to get this fixed via a scanner/etc that reads and relays out the file based structure.
_________________________________
Jeff Mohler mailto:jmohler@yahoo-inc.com
Tech Yahoo, Storage Architect, Principal
(831)454-6712 YPAC Gold Member
Twitter: @PrincipalYahoo CorpIM: Hipchat & Iris
On Friday, September 23, 2016 1:53 AM, Mike Gossett cmgossett@gmail.com wrote:
my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall.
The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.
On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland jordan.slingerland@gmail.com wrote:
Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it. I have specifically asked what stipulation. In a VDI environment with linked clones, etc. Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.
I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or what?
Any comments welcome.
--Jordan
______________________________ _________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/ mailman/listinfo/toasters http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
blockquote, div.yahoo_quoted { margin-left: 0 !important; border-left:1px #715FFA solid !important; padding-left:1ex !important; background-color:white !important; } Compaction is block content based. Not just the block. Snap mirror doesn't understand block contents.
Sent from Yahoo Mail for iPhone
On Friday, September 23, 2016, 10:25 PM, jordan slingerland jordan.slingerland@gmail.com wrote:
My understanding is that both are block based operations so it makes sense to me that the blocks would be put down on disk unchanged. Perhaps it has something to do with the fact that you may want to reverse the snapmirror back to the system that presumably does not have compaction enabled or support the feature, so the blocks are left uncompacted? If the vol move is within the same controller AFF the lack of backward compatibility is not a concern. Just a though, I don't know.
On Fri, Sep 23, 2016 at 10:57 AM, Francis Kim fkim@berkcom.com wrote:
Strange how compaction appears to work with vol move but not with snap mirror. . On Sep 23, 2016, at 4:39 AM, Steiner, Jeffrey Jeffrey.Steiner@netapp.com wrote:
A vol-move operation will cause compaction to happen. I know it's not ideal but at least its internal. Obviously a scanner is preferred but functionally it would be a lot like a vol move. Likewise if you use FLI to import a LUN that will trigger all the efficiency features during the import.
Sent from my mobile phone. On 23 Sep 2016, at 02:10, Jeffrey Mohler jmohler@yahoo-inc.com wrote:
Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else. (SSET diagnosis)
However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it.
It must transfer, as far as we're told today, via a host/file based migration. Be thinking bout this when you consider an AFF migration. It must be done outside on ONTAP.
We have pushed to get this fixed via a scanner/etc that reads and relays out the file based structure. ______________________________ ___Jeff MohlerTech Yahoo, Storage Architect, Principal(831)454-6712 YPAC Gold Member Twitter: @PrincipalYahoo CorpIM: Hipchat & Iris
On Friday, September 23, 2016 1:53 AM, Mike Gossett cmgossett@gmail.com wrote:
my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall. The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.
On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerlandjordan.slingerland@gmail.com wrote:
Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it. I have specifically asked what stipulation. In a VDI environment with linked clones, etc. Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.
I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or what?
Any comments welcome.
--Jordan
______________________________ _________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/ mailman/listinfo/toasters
______________________________ _________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/ mailman/listinfo/toasters
______________________________ _________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/ mailman/listinfo/toasters
______________________________ _________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/ mailman/listinfo/toasters
______________________________ _________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/ mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
blockquote, div.yahoo_quoted { margin-left: 0 !important; border-left:1px #715FFA solid !important; padding-left:1ex !important; background-color:white !important; } Good to know. Have not received that feedback yet from the team. But I've been in Thailand for a week.
Sent from Yahoo Mail for iPhone
On Friday, September 23, 2016, 6:37 PM, Steiner, Jeffrey Jeffrey.Steiner@netapp.com wrote:
A vol-move operation will cause compaction to happen. I know it's not ideal but at least its internal. Obviously a scanner is preferred but functionally it would be a lot like a vol move. Likewise if you use FLI to import a LUN that will trigger all the efficiency features during the import.
Sent from my mobile phone. On 23 Sep 2016, at 02:10, Jeffrey Mohler jmohler@yahoo-inc.com wrote:
Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else. (SSET diagnosis)
However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it.
It must transfer, as far as we're told today, via a host/file based migration. Be thinking bout this when you consider an AFF migration. It must be done outside on ONTAP.
We have pushed to get this fixed via a scanner/etc that reads and relays out the file based structure. _________________________________Jeff MohlerTech Yahoo, Storage Architect, Principal(831)454-6712 YPAC Gold Member Twitter: @PrincipalYahoo CorpIM: Hipchat & Iris
On Friday, September 23, 2016 1:53 AM, Mike Gossett cmgossett@gmail.com wrote:
my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall. The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.
On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerlandjordan.slingerland@gmail.com wrote:
Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it. I have specifically asked what stipulation. In a VDI environment with linked clones, etc. Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.
I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or what?
Any comments welcome.
--Jordan
______________________________ _________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/ mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Those sales folks may be misunderstanding the details. Ask to see the details of the guarantee, and read for yourself.
My experience with CDOT 8 on a AFF8020 was that video didn't compress/dedupe hardly at all (12% savings from dedupe, 0% from compression). Student home directories are getting about 48% savings (close to 2:1).
I imagine certain data stores will have much better savings (like lots of copies of VMs with identical OS images on them).
NetApp has a tool you can use to estimate savings of your own data, though it caps the size of the analysis at 2TB (regardless of the size of the volume being analyzed). They call it SSET (Space Savings Estimation Tool). I'm not sure whether that's specific to CDOT 8 or not.
Re:
From: jordan slingerland jordan.slingerland@gmail.com Date: Thu, 22 Sep 2016 14:32:30 -0400 Subject: AFF and 4:1 guaranteed efficiency To: toasters@teaparty.net
Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it. I have specifically asked what stipulation. In a VDI environment with linked clones, etc. Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.
I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or what?
Any comments welcome.
--Jordan
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
hose sales folks may be misunderstanding the details. Ask to see the details of the guarantee, and read for yourself.
I agree. Thanks for all the responses, much more inline with what I would think. My data is not video or audio, but I did bring that up to the sales guys and very specifically they said they could get 4:1 on it. In fact the only example dataset they had any hesitation on promising verbally was the output of /dev/urandom. I call BS. Don't get me wrong, ontap9 on AFF sounds like it contains a lot of great incremental improvement and I am excited to try one out. I just prefer realistic promises.
--Jordan
On Thu, Sep 22, 2016 at 3:07 PM, Brian Parent bparent@ucsd.edu wrote:
Those sales folks may be misunderstanding the details. Ask to see the details of the guarantee, and read for yourself.
My experience with CDOT 8 on a AFF8020 was that video didn't compress/dedupe hardly at all (12% savings from dedupe, 0% from compression). Student home directories are getting about 48% savings (close to 2:1).
I imagine certain data stores will have much better savings (like lots of copies of VMs with identical OS images on them).
NetApp has a tool you can use to estimate savings of your own data, though it caps the size of the analysis at 2TB (regardless of the size of the volume being analyzed). They call it SSET (Space Savings Estimation Tool). I'm not sure whether that's specific to CDOT 8 or not.
Re:
From: jordan slingerland jordan.slingerland@gmail.com Date: Thu, 22 Sep 2016 14:32:30 -0400 Subject: AFF and 4:1 guaranteed efficiency To: toasters@teaparty.net
Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it. I have specifically asked what stipulation. In a
VDI
environment with linked clones, etc. Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.
I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or
what?
Any comments welcome.
--Jordan
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
-- Brian Parent Information Technology Services Department IT Infrastructure Operations Group Workplace, Internal, Research, and Educational Platforms (WIRE) team UC San Diego (858) 534-6090
On Thu, Sep 22, 2016 at 03:21:02PM -0400, jordan slingerland wrote:
hose sales folks may be misunderstanding the details. Ask to see the details of the guarantee, and read for yourself.
I agree. Thanks for all the responses, much more inline with what I would think. My data is not video or audio, but I did bring that up to the sales guys and very specifically they said they could get 4:1 on it. In fact the only example dataset they had any hesitation on promising verbally was the output of /dev/urandom. I call BS. Don't get me wrong, ontap9 on AFF sounds like it contains a lot of great incremental improvement and I am excited to try one out. I just prefer realistic promises.
--Jordan
As other's have mentioned, it's likely more about matching up with the marketing of competitors. Just about everyone has a data reduction guarantee now with the same asterisks and strings attached.
Ray
As it was mentioned before, you can use SSET to measure efficiency of NetApp technologies with your dataset. Last version of SSET support inline data compaction. You can download SSET from Utility Toolchest http://mysupport.netapp.com/tools/index.html
And here is some information about the guarantee:
"The total effective storage capacity ratio is calculated based on compression, deduplication, compaction, snapshots and clones on the dataset.
Although all volumes (new and migrated ones) need to be thinly provisioned, thin provisioning is not included in measuring the total effective storage capacity ratio."
"The guarantee is for all workloads running on All Flash FAS systems. The AFF Systems can be added to the new cluster provided they adhere to the NetApp Storage Efficiency technologies. However, already compressed data and encrypted data is excluded from the guarantee. "
Just ask sales to show you program Terms and Conditions.
On Thu, Sep 22, 2016 at 10:27 PM, Ray Van Dolson rvandolson@esri.com wrote:
On Thu, Sep 22, 2016 at 03:21:02PM -0400, jordan slingerland wrote:
hose sales folks may be misunderstanding the details. Ask to see the details of the guarantee, and read for yourself.
I agree. Thanks for all the responses, much more inline with what I
would
think. My data is not video or audio, but I did bring that up to the
sales
guys and very specifically they said they could get 4:1 on it. In fact
the
only example dataset they had any hesitation on promising verbally was
the
output of /dev/urandom. I call BS. Don't get me wrong, ontap9 on AFF
sounds
like it contains a lot of great incremental improvement and I am
excited to
try one out. I just prefer realistic promises.
--Jordan
As other's have mentioned, it's likely more about matching up with the marketing of competitors. Just about everyone has a data reduction guarantee now with the same asterisks and strings attached.
Ray _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
I honestly don't know the terms of the guarantee at this time, but here's a few observations on this topic of efficiencies and guarantees:
1) Always read the remedies for the guarantees. They seem mostly the same to me across the vendors.
2) There's a reason for that - the efficiency rates are more or less the same for any of the various products on the market.
I've noted a few competitors who tend to cite 4:1, 5:1, or 6:1 without any qualification. On occasion, I've heard of one of them having to send lots of extra shelves of disks to make up for the shortfall, but most of the time the answer is "you had a nonstandard configuration" which is a different way of saying "too bad".
Most customers I know do see about 4:1 efficiencies when they properly implement an ONTAP 9 system, but obviously there will be variations. VDI is usually pretty easy because the VMDK's deduplicate extremely well. I've seen databases that were mostly empty and compressed and compacted 80:1. If someone is in the habit of storing loads of zeros in their databases, we'd blow that 4:1 out of the water, but that's a fringe case. It's due to compaction. All those 8K blocks are composed of a small header and trailer sandwiching some zeros. The zeros compress mostly out of existence and compaction will then store the residual header/trailer data into a tiny space. Fringe case, but not out of the question. On the flipside, I ran into a customer who was getting zero compression because their DBA's had enabled encryption across the board without telling the storage team. They were getting no savings at all. Still, 4:1 is about the typical level in my experience with ONTAP.
There's also more than just compression and efficiency. I know one customer who took a 60TB database and cloned it 40 times. That's 40:1 right there without compression or deduplication. You could do the same thing on some competing storage arrays and the clone savings would be done via deduplication, whereas we do it via direct cloning of a snapshot. It's quicker, but it's not classified as "efficiency" because it's not done via deduplication or cloning. It sometimes isn't getting the deserved credit as an efficiency option.
It's also worth mentioning the big picture. For one, the cost of SSD's keeps dropping, which erodes the value of efficiency. The cost savings are less and less all the time. Maybe some product out there does have 4:1 efficiency on a certain data set while ONTAP only had 3:1, but how much money does that really save, and what are the consequences? There are more features and needs beyond space savings. For example, some products use larger compression block sizes and get better efficiency. You could even force this with ONTAP via secondary compression which uses a larger block size and therefore delivers better efficiency levels. Unfortunately, with data sets like VDI and databases that involve small block overwrites you will see a latency hit. So, better efficiency with worse performance.
From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of jordan slingerland Sent: Thursday, September 22, 2016 1:33 PM To: toasters@teaparty.net Subject: AFF and 4:1 guaranteed efficiency
Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it. I have specifically asked what stipulation. In a VDI environment with linked clones, etc. Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1. I have A LOT of trouble believing that. So, 4:1 dedup guaranteed or what? Any comments welcome. --Jordan
-
Arthur Alikulov -
Brian Parent -
Cavin O'Quin -
Francis Kim -
Jeffrey Mohler -
jordan slingerland -
Klise, Steve -
Michael Bergman -
Mike Gossett -
Ray Van Dolson -
Steiner, Jeffrey -
Tim McCarthy -
tmac